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PREFACE 


When the JOBSTART Demonstration began nearly a decade ago, there was a growing 
realization of the employment problems ahead for young people without a high school diploma. 
Already, the severe decline in the inflation-adjusted earnings of school dropouts had begun, and an 
increasing proportion of young dropouts were outside the mainstream economy, neither working nor 
participating in skill-building activities. Since that time, the debate has intensified over how best to 
ease the transition into the workforce for those who might otherwise be unable to move readily from 
adolescence to eventual self-sufficiency. 

JOBSTART was implemented to produce evidence of the effectiveness of one approach. 
That strategy consisted of working within the main federal training program for disadvantaged youths 
— the Job Training Partnership Act (JTPA) system — to provide a combination of basic skills 
educatior,, occupational training, support services, and job placement assistance for low-skilled, young 
school dropouts. At the time JOBSTART began, this represented a departure from common JTPA 
practice, which emphasized short-term services for somewhat more employable clients. The 
demonstration was possible only because foundation, federal, and state and local funders were 
committed to experimenting with an alternative approach. 

Now, as the JOBSTART Demonstration ends, interest in programs to facilitate entry into the 
workforce is great. The new findings in this report, based on four years of follow-up, can help guide 
the search for better policy. While the final results of the demonstration are less positive than had 
been hoped for, they do provide a basis on which to build more effective programs. JOBSTART, 
unlike many other youth initiatives, did lead to some long-term gains in employment and earnings, 
which appear to continue over time. For some subgroups and sites, the results are strongly positive. 
The challenge is to build on these results. We hope that the implications of the implementation and 
impact findings and policy suggestions explored in this report are an important beginning of that 
effort. 


Judith M. Gueron 
President 
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This report, which completes the JOBSTART Demonstration, addresses issues closely linked 
to the nation’s ongoing debate about how best to improve the employment and earnings prospects 
of low-skilled, economically disadvantaged young people, whe otherwise will live outside the economic 
mainstream. There is compelling evidence that youths who have dropped out of high school are 
increasingly unable to find a job, much less a job that supports a decent standard of living. The 
statistics are stark: In 1992, more than half of all 16- to 24-year-olds who had dropped out of school 
did not work during the year. For blacks, the figures are even more discouraging, with less than 30 
percent working. The results from past studies of initiatives to combat these problems have generally 
been negative or inconclusive. There is little solid evidence about what works. 

The JOBSTART Demonstration was an unusual collaborative effort to provide such evidence. 
The demonstration — developed and evaluated by the Manpower Demonstration Research 
Corporation (MDRC) — was implemented between 1985 and 1988 in 13 sites ranging from 
community-based organizations to schools to Job Corps Centers. In each site, 17- to 21-year old, 
economically disadvantaged school dropouts with poor reading skills participated in education and 
vocational training, and received support services and job placement assistance. In many ways, this 
initiative drew on lessons from the residential Job Corps program, which provides similar — though 
more intensive — services and, in an influential study, was found to raise young people's earnings and 
to be cost-effective for taxpayers. Operating funds for the JOBSTART Demonstration came 
primarily from the Job Training Partnership Act of 1982 (JTPA), which supports the nation’s 
principal employ.nent and training program for economically disadvantaged people. 


Overview of the Findings 


The demonstration results presented in this report are based on a comparison of young people 
who were randomly assigned either to a program, or “experimental,” group (given access to 
JOBSTART services) or to a control group (who were not). The difference (often called the 
“impact") that the program made over time in key outcomes such as educational attainment, 
employment, earnings, and welfare receipt was estimated by comparing the experiences of the 
experimental and control groups. The main findings, based on four years of follow-up data, include 
the following: 
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¢ The local sites were able to recruit the target group of seriously disadvantaged 
youths and implement the four core JOBSTART components: education, 
occupational training, support services, and job placement assistance. 


* More than 90 percent of the experimental group participated in JOBSTART, and 
they averaged 400 hours of activities, although there was wide variation in the 
intensity of their participation. In addition, over the four years of follow-up, the 
members of both the experimental and control groups averaged 400 hours of 
participation in non-JOBSTART activities. The impact estimates in this report 
represent the incremental effect of JOBSTAR TI services over the level of services 
received by controls, and although youths in the control group received a fair 
amount of services, those in the experimental group received substantially more. 


* JOBSTART led to a significant increase in the rate at which the youths passed 
the GED (General Educational Development) examination or completed high 
school. Overall, 42 percent of those in the experimental group attained this 
milestone, as compared to 28.6 percent in the control group, and similar impacts 
were present for most key subgroups of the sample. For all groups, most of the 
increase in educational attainment came through receipt of a GED rather than 
completion of high school. 


¢ As expected, youths in the experimental group earned less on average than those 
in the control group during the first year of follow-up. In the final two years of 
follow-up, experimentals’ earnings appeared to overtake those of controls for the 
full sample (by approximately $400 per year), and there were similar patterns for 
several subgroups. In most cases, however, the magnitude of these impacts was 
disappointing and they were not statistically significant according to the usual 
tests. 


¢ Encouraging earnings impacts include those for young men who had been arrested 
between age 16 and program entry ($1,129 and $1,872 in years three and four of 
follow-up, respectively), and for young men and women who had dropped out of 
school because they had educational difficulties ($726 and $592 in the last two 
years of follow-up). 


¢ Earnings impacts were very large for one site in the demonstration: the Center 
for Employment Training (CET) in San Jose, California. Earnings impacts at 
CET/San Jose in the last two years of follow-up totaled more than $6,000, far 
larger than at any other site. When these results are combined with CET/San 
Jose’s strong earnings impacts in the Minority Female Single Parent (MFSP) 
Demonstration, there is growing evidence of the strength of the program at this 
site. 


© Overall, JOBSTART led to little change in youths’ receipt of public assistance, 
although there was a notable positive finding: Young women in the experimental 


ISee John Burghardt et al., Evaluation of the Minority Female Single Parent Demonstration, Vol.1, Summary 
Report (New York: The Rockefeller Foundation, 1992). 
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group who were not mothers when they entered the program were significantly 
less likely than their control group counterparts to receive AFDC (Aid to Families 
with Dependent Children) during the later years of follow-up.” 


¢ From the perspectives of taxpayers and society as a whole, the investment in 
JOBSTART services — about $4,500 per experimental — was not repaid through 
increases in experimentals’ earnings or other quantified benefits by the end of the 
follow-up period; the initial earnings losses were too large and the later payoffs 
too modest for this to occur. 

The experience of JOBSTART provides a number of lessons for building future youth 
programs. Unlike several other youth initiatives that have been studied, JOBSTART did produce 
apparent impacts in the period after program participation ("apparent” because they just missed 
passing the usual tests for statistical significance), and these impacts do not appear to decline over 
time, although the trend is somewhat unclear in the last four months of follow-up. The central 
problems were the substantial earnings losses during program participation for some subgroups and 
the modest earnings gains in later years for most subgroups. The final section of this summary (and 
the last chapter of the full report) discusses suggestions for lessening initial losses and increasing later 
payoffs when operating similar programs. 


The Goals of the JOBSTART Demonstration 


As mentioned previously, JOBSTART was modeled after the residential Job Corps. However, 
the residential Job Corps cannot be offered to most young dropouts because its comprehensive 
services are relatively expensive, it operates in specialized centers, and only young people willing and 
able to live away from home can participate. JOBSTART drew on the Job Corps’ experience by 
offering several of the same basic components in a nonresidential program. Some hallmarks of the 
Jobs Corps — includiny extensive support services and financial compensation — were not available 
in most demonstration sites because of severe restructions in JTPA, the primary funding source for 
JOBSTART programs. For the same reason, paid work experience, which is another important 
element in many Job Corps Centers, was not included in JOBSTART programs. Amendments to 
JTPA enacted in 1992 have eased this situation. 

The JOBSTART Demonstration — focused on improving the lives of young, low-skilled school 
dropouts — sought answers to five key policy questions relating to its general programmatic approach. 


2More precisely, this group includes young women who either were not mothers or were not living with 
their children at the time they entered the program. 


-xix- 


<0) 


© Recruitment. Could local agencies attract young, economically disadvantaged, 
low-skilled school dropouts into an alternative education and traiaing program? 

Many program operators have discovered how difficult it is to reach alienated young people 
and provide them with the support they need to participate in an intensive and challenging program 
such as JOBSTART. 

¢ Implementation. Could sites put in place a package of services designed to 

address the needs of these youths while working within the constraints of JTPA 
funding, performance standards, and administrative practices? 

Sites participating in JOBSTART had to raise operating funds from existing public funding 
sources, and most relied on Title ILA of JTPA. When JOBSTART began, regulations and prevailing 
administrative practices encouraged local JTPA agencies to emphasize shorter-term, lower-cost 
programs and to enroll participants who were more employable than the JOBSTART target group. 
Thus, the successful implementation of JOBSTART could not be taken for granted within JTPA. 

¢ Participation. Would the young people respond favorably to this opportunity and 

make an investment of their time and effort by participating in the services? 

The conditions that make it difficult to recruit low-skilled youths into education and training 
programs often preclude their completing the coursework. Many are reluctant to return to a school 
setting and/or require extensive support services because of child care responsibilities, housing 
problems, or other disruptions in their lives. The financial pressures they face are severe and it is 
difficult to attract youths into programs that cannot pay them stipends, as is the case under JTPA 
tules. 
¢ Impacts on educational attainment. Would the program lead to an increase in 


educational attainment, as measured by receipt of a high school diploma or 
GED? 


¢ Impacts on employment, earnings, and other outcomes. Would the program lead to 

increased employment and earnings, and impacts on other outcomes? 

JOBSTART's major goal was to increase the employment and earnings and reduce the welfare 
receipt of young, low-skilled school dropouts. Increased educational attainment, primarily receipt of 
a GED, was primarily seen as a vehicle to improve the employment prospects of youths. 

The JOBSTART Demonstration provided a rigorous test of the difference that JOBSTART's 
combination of nonresidential services made in both the educational attainment and labor market 
success of young people. Youths who applied for the program were randomly assigned to a group 


given access to JOBSTART (the experimental group) or to a group not given that access but free 
to seek other services in the community (the control group). Since the two groups were created by 
chance, using a lottery-like process, there was only one systematic difference between them: Only 
those in the experimental group could receive JOBSTART services. The control group provides 
information on what those in the experimental group would have done if there had been no 
JOBSTART program: Some would have found alternative services, some would have worked, and 
so forth. Therefore, a comparison of the two groups’ behavior over time provides an estimate of the 
difference that the added services the experimental group received made in their subsequent 
educational attainment, employment, earnings, welfare receipt, and other outcomes. 

Importantly, given the diversity of the JOBSTART youths and their program experience, the 
demonstration’s research design also provided a framework for impact estimates for subgroups of the 
full sample. These involve comparisons of members of the experimental group and of the control 
group with the same initial characteristics. Important subgroups highlighted in this summary include: 
men, women who were living with children of their own at entry into the program (for brevity, 
“custodial mothers”), all other women in the sample, men arrested between age 16 and program entry, 
and youths who dropped out of school for reasons related to their educational performance or 
experience (rather than because they wished to work or for other reasons). The gender-based 
subgroups are important because past research suggests that impacts might vary between women and 
men owing to their differences in prior work experience and child care responsibilities. Young men 
with an arrest record were highlighted because of the need to develop means to prevent further 
criminal activity among this group. Those who left school because of educational problems were 
analyzed separately to understand if JOBSTART could serve as an alternative to regular schools for 
these youths. 


The Structure of the JOBSTART Demonstration 


The JOBSTART Demonstration guidelines, developed by MDRC and other experts, specified 
the target group and the characteristics of the core service components. The local programs were 
to recruit 17- to 21-year-old, economically disadvantaged school dropouts who read below the eighth- 
grade level and were eligible for JTPA Title IIA programs or the Job Corps (which is funded under 
Title IVB of JTPA). The four central program components were to be implemented as follows: 

* Instruction in basic academic skills, based on individualized curricula chosen by 


the sites to allow youths to proceed at their own pace toward competency goals 
in reading, communication, and basic computational skills. 
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* Occupational skills training provided in a classroom setting combining theory and 
hands-on experience to prepare participants for jobs in high-demand occupations. 


* Training-related support services including assistance with transportation and 
child care, counseling, and, where possible, additional support such as work- 
readiness and life skills (practical, everyday knowledge) training and needs-based 
or incentive payments tied to program performance. 


¢ Job placement assistance to help JOBSTART youths find training-related jobs. 


Sites were required to offer at least 200 hours of basic education and at least 500 hours of 
occupational training. Some sites were able to provide all four JOBSTART components themselves, 
while others had to serve as brokers for the young people, helping them gain entry into services at 
other agencies. 

Within this general framework, the 13 local JOBSTART programs (listed in Table 1) did vary, 
reflecting their diverse operating experiences, funding sources, clientele, and local service networks. 
Among the important types of local variation were: agency type (school, community-based 
organization, or Job Corps Center); the nature of the linkage between education and training (that 
is, whether they were offered sequentially or concurrently); in concurrent sites, the extent of 
integration of these two types of instruction; the stability of funding and program operations during 
the demonstration; and the strength of the implementation of the core JOBSTART components, 
especially training and job placement assistance. 

The 2,312 youths who applied for JOBSTART and were judged eligible were randomly assigned 
to either the experimental or control group. Follow-up surveys attempted to reach all members of 
both groups 12, 24, and 48 months after they were randomly assigned. This analysis of program 
participation and impacts uses a sample of 1,941 youths (84 percent of all those who were randomly 
assigned) for whom 48 months of follow-up data were available. Impacts reported in the following 
sections are statistically significant (that is, unlikely to have arisen by chance) unless otherwise noted. 


Participation Findings 

Young people in the JOBSTART sample could have improved their job-related skills by several 
means: through participating in JOBSTART (for those in the experimental group) or in other 
education and training programs, or by learning on the job. Program impacts were measured by 
comparing the experiences of the experimental and control groups; in essence, the impacts are a 
comparison of the payoff of the investment made by experimentals to that made by controls. 

The most likely means for JOBSTART to make a difference in the lives of the young people 
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TABLE 1 


THE JOBSTART SITES 


Agency Name Type of Prior Service JOBSTART 

and Location Organization Emphasis® Program Structure? 
Allentown Youth Services Community-based Education Sequential/brokered 
Consortium, Buttalo, NY° 

Atlanta Job Corps, Job Corps Center Education and Concurrent 

Atlanta, GA training 

Basic Skills Academy (BSA), Community-based Education Sequential/orokered 
New York, NY 

Capito! Region Education Community-based Education Sequential/orokered 
Council (CREC), 

Harttord, CT 

Center for Employment Community-based Training with Concurrent 
Training (CET), some education 

San Jose, CA 

Chicago Commons Association's Community-based Training Concurrent 
Industrial and Business 

Training Programs, 

Chicago, IL 

Connelley Skill Learning Adult vocational Education and Concurrent 
Center, Pittsburgh, PA school training 

East Los Angeles Skills Adult vocational Education and Concurrent 
Center, Monterey Park, CA school training 

El Centro Community College Community college Education and Sequential/in-house 
Job Training Center, training 

Dallas, 

Emily Griffith Opportunity Adult vocational Education and Concurrent 

School (EGOS), Denver, CO school training 

Los Angeles Job Corps, Job Corps Cemer Education and Sequential/in-house 
Los Angeles, CA training 

Phoenix Job Corps, Job Corps Center Education and Concurrent 
Phoenix, AZ training 

SER/Jobs for Progress, Community-based Training Concurrent 
Corpus Christi, TX 


NOTES: “Education refers to basic education, often as preparation for the GED examination. ‘Training® refers to 
instruction in occupational skills needed for spectic jobs. 
‘Concurrent programs offer basic education and occupational training concurrently from the beginning of 


componens 
8 broker tor occupational training, referring 
Cin October 1990 this site was renamed The Clarkson Center, inc. 
Sin September 1988 this site was renamed the Edmund J. Kahn Job Training Center. 
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was for those in the experimental group to participate in substantially more education and training 
activities than those in the control group, who had access to other services in the community or could 
be working. 

* Young people in the experimental group attended an average of 400 hours of 
JOBSTART activities, but behind this average there is great diversity of 
participation. 

These reported average total hours (which include zero hours for the 11 percent of 
experimentals who did not participate at all) were primarily time spent in education and occupational 
training (an average of 362 hours in these activities), as opposed to other activities such as life skills 
training. The intensity of JOBSTART varied among the sample: While 33 percent of experimentals 
were active for more than 500 hours, slightly more than 40 percent participated for 200 hours or less. 

The average length of stay in JOBSTART was 6.8 months, with 16 percent of experimentals 
still active in the program 12 months after random assignment and nearly 10 percent active 15 months 
after random assignment. This means that for most experimentals the first year of follow-up was 
primarily a period of program participation and that for 16 percent the second year also included 
months with program activity. This duration of participation was twice JTPA’s average duration of 
service for young dropouts during the same period and virtually the same as that of the Job Corps. 

© During the initial year of follow-up, more than three times as many 


experimentals as controls were active in some type of education or training. 
Though experimentals’ participation dropped sharply in subsequent years, for 
the entire four-year period there remained a clear difference in participation. 

Table 2 shows experimental-control differences in the percentage of youths who ever 
participated in education or training; the differences are shown for the full sample and key subgroups. 
In the four years following random assignment, 94 percent of experimentals versus 56.1 percent of 
controls received some type of education or training. Experimentals averaged 800 hours in these 
activities (not shown in the table), whereas controls averaged 432 hours. The differences were largest 
during the early months of the follow-up period, when most experimentals were active in 
JOBSTART, and gradually disappeared by the end of the first two years. All the subgroups that were 
analyzed showed large differences between experimentals and controls in service receipt. 


Impacts on Educationa) Attainment 
JOBSTART succeeded in raising the rate of educational attainment. 


TABLE 2 


IMPACTS ON RECEIPT OF EDUCATION OR TRAINING AND ON EDUCATIONAL ATTAINMENT 
THROUGH YEAR FOUR FOR THE FULL SAMPLE AND KEY SUBGROUPS 


Sample Experimentals Controls 
Outcome and Subgroup Size (%) (%) Difference 


Ever received any education 

or training by end of year 4 
Full sample 1,941 94.0 56.1 37.9°** 
Men 900 94.3 51.3 43.0°** 
Custodial mothers 508 95.0 59.7 35.3°** 
Other women 533 93.2 60.9 32.3°** 

Received GED or high school 

diploma by end of year 4 
Full sample 1,941 42.0 28.6 13.4¢** 
Men 900 42.0 28.3 1s.7""* 
Custodial mothers 508 42.0 26.7 15.3°** 
Other women 533 41.6 31.3 10.4** 


NOTES: Calculations for this table used data for all sample members for whom there were 48 
months of follow-up survey data, including those with values of zero for outcomes and those who 
were assigned to JOBSTART but did not participate. 

Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; * = 10 percent. 


¢ JOBSTART led to a substantial increase in the rate of GED certification or 

receipt of a high school diploma, with most who acquired one of these 
educational credentials obtaining a GED in the first two years of follow-up. 

Table 2 also presents JOBSTART's impacts on educational attainment during the 48 months 
of follow-up, again for the full sample and key subgroups. By the end of the four years, 42 percent 
of experimentals had obtained a high school diploma or GED, as compared to 28.6 percent of 
controls, for a statistically significant difference of 13.4 percentage points. This impact was similar 
to the findings in an evaluation of the residential Job Corps. 


¢ These educational attainment impacts were large for all the subgroups studied. 


Among men, 42 percent of experimentals versus 28.3 percent of controls completed high school 
or passed the GED examination during the follow-up period, for an impact of 13.7 percentage points 
(see Table 2). Among custodial mothers, the figures were 42 percent of experimentals and 26.7 
percent of controls, for an impact of 15.3 percentage points; and among all other women in the 
sample, the figures were 41.6 percent of experimentals and 31.3 percent of controls, for an impact 
of 10.4 percentage points. Numerous other subgroups defined by work experience, welfare receipt, 
prior education, initial reading level, reason for dropping out of school, prior criminal record, and age 
all showed similar large impacts. 


Labor Market Impacts for the Full Sample 


One of the central questions of the JOBSTART Demonstration was whether higher 
educational attainment would translate into greater employment and earnings for the experimental 
group. The theory behind JOBSTART was that the youths’ initial investment in the program would 
in the long run lead to increases in their employment and earnings. Figure 1 shows the expected 
relationship between the earnings of experimentals and controls over time. During the initial period 
of the JOBSTART Demonstration, the earnings of experimentals — who were then active in the 
program — were likely to be less than those of controls. The foregone earnings owing to 
participation in JOBSTART are represented in Figure 1 as the shaded “opportunity cost” during the 
early months of follow-up. Once their participation in JOBSTART ended, experimentals could move 
into employment, gradually catching up with and overtaking the earnings of the control group. The 
earnings gains in the later months were expected to be the payoff to participants from their earlier 
investment in the program. 


A THEORETICAL VIEW OF THE PAYOFF 
OF A PERSONAL INVESTMENT IN EDUCATION AND TRAINING 


Aver 
Eamings ($) 


, Wg 
_ 
| ——_——_ 


° As expected, more youths in the contro] group than in the experimental group 
worked during the first year of follow-up. In the second year, slightly more 
experimentals than controls worked, and in the third and fourth years the 
percentage working in each group was approximately the same. 

As shown in Table 3, 56.5 percent of experimentals and 60.8 percent of controls worked at 
some time during the first year of follow-up, for a 4.3 percentage point decrease in employment 
among experimentals relative to controls. In contrast, during the second year 67.5 percent of controls 
and 71 percent of experimentals ever worked, for a 3.5 percentage point positive impact. In years 
three and four, the employment rates for experimentals and controls were similar. 

¢ As expected, in the early part of the follow-up period experimentals earned 

significantly less than controls. 

As expected, experimentals earned less than controls during the first year of follow-up (see 
Table 3); this $499 difference was a clear opportunity cost of participating in the program. In the 
second year, although the proportion of experimentals working drew even with the proportion of 
controls, experimentals continued to lag slightly behind controls in hours worked per week and weeks 
worked per month. As a result, the earnings of experimentals remained slightly below those of 
controls during the second year; however, the difference ($121) was no longer statistically significant. 
The cumulative opportunity cost in the form of foregone earnings was, therefore, $620 at the 
midpoint of the follow-up period. 

© Ip the last two years of follow-up, experimentals appeared to earn more than 

controls, although the differences just missed being statistically significant 
under the usual tests. 

In the third year of follow-up, the average earnings of experimentals rose sharply to exceed 
those of controls by $423, an impact very close to statistical significance under the usual tests. This 
earnings gain persisted in the fourth year (declining slightly to $410), so that at the four-year point 
average total earnings for experimentals were $214 above those for controls. This four-year impact 
was not statistically significant, and the trend in earnings impacts is unclear in the last four months 
of follow-up, as shown in Figure 2. The apparent earnings gains in the last two years of follow-up 
are the result of small positive impacts on hours worked and wages (not shown in Table 3). 


Many past studies of nonresidential education and training programs have found starkly 
different results for men and women. Thus, it is important to move behind the findings for the full 
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TABLE 3 


IMPACTS ON EMPLOYMENT, HOURS OF WORK, AND EARNINGS 
THROUGH YEAR FOUR FOR THE FULL SAMPLE 


Outcome and 
Follow-Up Period Experimentals Controls Difference 
Ever employed (%) 
Years 1-4 86.4 86.0 0.4 
Year 1 56.5 60.8 -4,.3°* 
Vear 2 71.0 67.5 3.5* 
Year 3 61.8 61.5 0.3 
Year 4 4 65.7 64.5 1.3 
Total hours worked 
Years 1-4 3,031 3,071 40 
Year 1 441 550 -109*** 
Year 2 760 775 “15 
Year 3 899 855 ag 
Year 4 930 890 40 
Total eamings ($) 
Years 1-4 17,010 16,796 214 
Year 1 2,097 2,596 -499*** 
Year 2 3,991 4,112 -121 
Year 3 §,329 4,906 423 
Year 4 5,592 5,182 410 
Sample size 988 953 


NOTES: Calculations for this table used data for all sample members for whom there 
were 48 months of follow-up survey data, including those with values of zero for outcomes 


Participate. 
Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; 
* = 10 percent. 


=-xxix- 


FIGURE 2 
MONTHLY EARNINGS FOR THE FULL SAMPLE, BY RESEARCH STATUS 


Total Eamings Per Month ($) 
8 8 3 


2 4 6 8 10 12 14 16 18 20 22 24 26 26 30 32 M4 3 3B 40 42 44 46 48 
Months Since Random Assignment 


NOTE: Calculations for this figure used data for all sample members for whom 
there were 48 months of follow-up survey data, including those with values of 
zero tor outcomes and those who were assigned to JOBSTART but did not participate. 


sample and examine the effects of JOBSTART on specific subgroups. This report focuses on three 
subgroups defined by gender and child care responsibilities: men (more than 85 percent of whom 
were not custodial parents when they entered the program), custodial mothers, and all other women 
in the sample. It also highlights impacts for men arrested between age 16 and program entry and for 
youths (both male and female) who dropped out of school for school-related reasons. In these 
subgroup analyses, sample sizes were of course smaller than for the full sample; therefore, statistically 
significant findings were less likely. Reflecting this limitation, most of the subgroup impacts reported 
in this and succeeding sections are not statistically significant using standard tests and should be 
viewed as suggestive but not definitive evidence. Despite this qualification, it is important to explore 
these findings because there is great policy interest in certain subgroups and because JOBSTART 
did result in positive — and significant — impacts on some outcomes for some groups of young 
people. 

¢ For both groups of women, after an initial period of small earnings losses, 

impacts in the last two years of follow-up appeared to be slightly positive. 

Table 4 shows employment and earnings impacts for the three gender-based subgroups. For 
custodial mothers (the top panel), the employment rate of experimentals in the first year of follow-up 
was slightly above that of controls, although controls earned slightly more. In the second year, a 
significantly higher percentage of experimentals worked, but their earnings were only slightly higher 
than those of controls. 

In the remaining two years, the employment rates of experimentals and controls were 
approximately equal and experimentals again appeared to earn more ($328 more in year three and 
$290 in year four). For the entire four-year period, experimentals earned $625 more than controls. 

For all other women (the middle panel), a slightly higher percentage of controls than 
experimentals worked at some point in the first year, while in the second year a higher percentage 
of experimentals worked. Reflecting this pattern, the earnings of controls exceeded those of 
experimentals in the first year; in the second year, ex erimentals’ earnings drew even with controls’. 
In the third and fourth years of follow-up, experimenta's appeared to pull ahead of controls (by $420 
in year three and $461 in year four). 

© Men initially experienced large earnings losses, but in the last two years of 

follow-up earnings impacts appeared to be somewhat positive. 

Among men (the bottom eet of Table 4), a significantly lower percentage of experimentals 
than controls worked at some point in the first year, but in the remaining three years employment 
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TABLE 4 


IMPACTS ON EMPLOYMENT AND EARNINGS THROUGH YEAR FOUR, 
BY GENDER AND PARENTAL STATUS 


3 Ss , Outcome, 
| and Follow-Up Period Experimentals Controls Difference 


Custodial mothers 
Ever employed (%) 
Years 1-4 


rates were nearly equal. Earnings for experimentals were significantly below those for controls in the 
first year (by $812), but the gap narrowed somewhat in the second year to $396. The earnings of 
male experimentals exceeded those of controls by $444 in the third year and $492 in the fourth year. 
¢ The employment experiences of the controls in these three subgroups provide 
much of the explanation for the pattern of impacts during the early follow-up 
period. 

One likely explanation for better initial earnings results for women is that it is easier to improve 
the employment and earnings of those who do not spend much time in the world of work (for 
example, young mothers) than of those who are already in the labor force but fail to find and keep 
steady, well-paying jobs (for example, poorly skilled young men). Thus, from this perspective, women 
have greater potential to improve their labor market outcomes than do men, and less to lose (in 
terms of foregone employment and earnings) by investing in education and training. And among 
women, those who are caring for children are likely to have the least prior employment experience 
and foregone earnings. 

The JOBSTART sample followed this pattern, as shown in Table 4. During the first year after 
random assignment, 74.9 percent of male controls worked at some point, compared to 38.8 percent 
of custodial mothers and 57.8 percent of other women in the control group. During this period, when 
many experimentals were participating in the program, the impact on employment rates was 2.2 
percentage points for young mothers, -2.2 percentage points for other women, and -9.2 percentage 
points for men. Men and women who were not living with children of their own apparently paid an 
opportunity cost for JOBSTART participation in terms of foregone employment, while young mothers 
(whose control group counterparts were much less likely to be working) did not. At the two-year 
point, custodial mothers in the experimental group had already virtually erased their initial earnings 
loss, while experimentals among the other women remained $267 behind their control group 
counterparts. For men, earnings impacts were still negative in the second year of follow-up. 

Table 5 shows earnings impacts for two other important subgroups. 

© Men arrested between age 16 and program entry had large positive earnings 

impacts in the last two years of follow-up and for the entire four-year period. 

Among the small sample of men with an arrest prior to entering JOBSTART, experimentals 
initially experienced earnings losses, but in the second year of follow-up, impacts turned positive and 
continued to grow. In the third year the earnings impact was $1,129, and in the fourth it was $1,872 
(or 37 percent of the control earnings base), a statistically significant impact even with the small 
sample. For the entire four-year follow-up period, earnings impacts totaled $2,491. 
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TABLE 5 
IMPACTS ON EARNINGS THROUGH YEAR FOUR 
FOR OTHER KEY SUBGROUPS 
Subgroup and Experimentais Controls Difference 
Follow-Up Period (S) (S) (S) 
Men arrested between age 16 
and random assignment 
Years 1-4 22,835 20,344 2,491 
Year 1 3,091 4,027 -936 
Year 2 5,722 5,297 425 
Year 3 7,052 5,098 1,129 
Year 4 6,970 5,098 1,872** 
Sample size 127 110 
Youths who left regular high school 
for school-related reasons 
Years 1-4 17,590 16,409 1,181 
Year 1 2,179 2,491 312 
Year 2 4,287 4,112 175 
Year 3 5,486 4,760 726* 
Year 4 5,638 5,046 592 
Sample size 489 436 


NOTES: Calculations for this table used data for all sample members for whom there were 48 
months of follow-up survey data, including those with values of zero for outcomes and those who were 
assigned to JOBSTART but did not participate. 

Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; * = 10 percent. 
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JOBSTART thus was an effective means to increase the legitimate earnings of young men who 


had already had an encounter with the criminal justice system. Since much urban crime is committed 
by young men who have prior arrests, finding that JOBSTART opened up new opportunities for this 
group is encouraging. 

* Youths who had dropped out of school for educational reasons had positive 

impacts on earnings in the last two years of follow-up. 

Those JOBSTART youths who Icft school for reasons such as poor grades, dislike of school, 
and discipline problems experienced a relatively small first-year earnings loss ($312), as shown in the 
bottom panel of Table 5. In contrast, those who left for employment-related reasons (not shown in 
the table) experienced a loss of $1,108. In the third and fourth years of follow-up, earnings impacts 
for those who had left school for school-related reasons were $726 and $592, respectively, with the 
third year being statistically significant. Earnings impacts for this group for the entire four-year 
period were $1,181, in contrast to negative earnings impacts for those leaving school for employment- 
related and other reasons. 

This suggests that JOBSTART worked better for young people who were "pushed out" of 
regular school because of problems in that educational environment rather than “pulled out" by a 
desire to work or pressing problems outside the school setting. Apparently JOBSTART succeeded 
in creating an alternative educational setting that “felt different" from regular high school and could 
make a difference for young people who had serious problems in a traditional setting. 


Impacts on Other Outcomes 


A program such as JOBSTART could also have impacts on outcomes such as welfare receipt, 
pregnancy and birth rates, marriage rates, criminal activity, and drug use. For these outcomes, the 
main findings indicate an encouraging pattern of impacts for women who were not custodial mothers 
at entry into JOBSTART and for men who had been arrested between age 16 and entering the 
program. 

¢ For women who were not custodial mothers at entry into JOBSTART, there was 

a consistent pattern of reductions in AFDC receipt and payments, and many of 
these impacts were statistically significant. 

The bottom panel (“other women") of Table 6 shows impacts on welfare receipt and payments 
and on pregnancy and childbearing for this group. Only a small percentage of these women received 
AFDC in the first year of follow-up (12.6 percent of experimentals and 14.9 percent of controls), but 
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TABLE 6 


IMPACTS ON AFDC, PREGNANCY, AND CHILDBIRTH THROUGH YEAR FOUR, 
BY PARENTAL STATUS AT THE TIME OF RANDOM ASSIGNMENT 


Subgroup, Outcome, 


and Follow-Up Period Experimentals Controls Difference 
Custodial mothers 
Ever received AFDC (%) 
Years 1-4 84.8 81.6 3.2 
Year 1 65.6 61.2 44 
Year 2 75.5 74.3 1.2 
Year 3 §7.0 §7.5 0.5 
Year 4 59.3 60.5 “1.2 
Total AFDC income (S$) 
Vears 1-4 9,371 9,334 37 
Year 1 2,167 2,072 95 
Year 2 2,402 2,279 123 
Year 3 2,310 2,343 33 
Year 4 2,493 2,641 -148 
Ever pregnant (%) 
Years 1-4 76.1 67.5 6.6** 
Ever gave birth (%) 
Years 1-4 67.8 §7.9 9.9°* 
Other women 
Ever received AFDC (%) 
Years 1-4 38.0 45.1 7.1 
Year 1 12.6 14.9 2.3 
Year 2 22.0 29.9 -8.0°* 
Year 3 24.1 33.2 -9.1°* 
Year 4 30.7 39.3 -8.6* 
Total AFDC income (S$) 
Years 1-4 3,204 3,979 -775 
Year 1 312 308 4 
Vear 2 604 795 -191 
Year 3 1,001 1,311 -310* 
Vear 4 1,287 1,564 -277 
Ever pregnant (%) 
Years 1-4 64.4 65.6 “1.2 
Ever gave birth (%) 
Vears 1-4 52.7 56.5 3.9 


NOTES: Calculations for this table used data for all sample members for whom there 
were 48 months of follow-up survey data on the specific outcome, including those with 
values of zero for outcomes and those who were assigned to JOBSTART but did not 
participate. 
Statistical significance levels are indicated as *** = 1 percent; ** = 5 
percent; * = 10 percent. 
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the percentage receiving aid among both experimentals and controls gradually grew over time as over 
half became mothers. However, the increase in welfare receipt among controls was greater than 
among experimentals, with a growing difference in AFDC receipt rates during years two through four. 
This also led to a smaller average AFDC grant for experimentals compared to controls, especially in 
the third and fourth years. (Note that the figures in the table include zero dollars for those who did 
not receive AFDC.) . 


These findings on AFDC receipt and payments may be partly related to the findings on 
pregnancy and birth rates for this subgroup over the four-year follow-up period. Although the rate 
at which these women gave birth during the follow-up period was high (over 50 percent), the rate 
for experimentals was slightly lower than for controls. 


¢ For women who were custodial mothers when they entered JOBSTART, impacts 
were much less encouraging, with significantly increased childbearing (most 
noticeably among mothers who had been married at entry into the program) and 
no impacts on AFDC receipt. 


The top panel of Table 6 shows that for women who were custodial mothers at entry into 
JOBSTART there were no significant impacts on either the receipt of AFDC or the average amount 
received. Experimentals in this subgroup also had significantly higher rates of pregnancy (76.1 
percent versus 67.5 percent for controls) and birth (67.8 percent versus 57.9 percent for controls) 
over the four-year period. Impacts on pregnancy and childbirth were particularly high for custodial 
mothers who had been married when they entered JOBSTART; for this small group, pregnancy and 
births among experimentals were more than 20 percentage points higher than among controls. 

¢ For men arrested between age 16 and program entry, JOBSTART led to 

reductions in arrests in the post-program period (years two through four) and 
in drug use in year four, although the small sample size means that these 
impacts were generally not statistically significant. 

Table 7 shows arrest and drug use rates for young men who had been arrested between age 
16 and program entry. For all of the post-program outcomes, there was a consistent pattern of 
reductions in illegal behavior. Most notable was the significant impact on the use of drugs other than 
marijuana, where the level of use by experimentals was less than half that of controls. There 
appeared to be a small decline in arrests over the entire follow-up period for these young men, but 
the level of arrests during the four years among both experimentals and controls illustrates the 
continuing problems they had with the criminal justice system. 

While most of these impacts are not statistically significant, when combined with the strong 
earnings impacts reported earlier they provide evidence that JOBSTART served as an opportunity 
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TABLE 7 


IMPACTS ON ARRESTS AND DRUG USE 


FOR MEN ARRESTED BETWEEN AGE 16 AND RANDOM ASSIGNMENT 


Outcome and 
Follow-Up Period 


Ever arrested 
Year 1 
Years 1-4 
Ever used any drug in year 4 


Ever used any drug in year 4, 
excluding marijuana 


Ever used marijuana in year 4 


(%) 


35.1 
68.9 


25.3 


als Controls 
(%) 


35.1 
74.8 


30.2 


Difference 


<0. 1 
5.8 


€6.9° 
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NOTES: Calculations for this table used data for all sample members for whom there were 


participate. 
Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; * = 10 percent. 
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to change the lives of young men with a prior arrest, a group at great risk of falling into a pattern 
of behavior destructive to themselves and their communities. 

¢ JOBSTART led to a short-term reduction in arresis during the first year of 

follow-up for the full sample and some key subgroups. 

In the initial year of follow-up, when most experimentals were active in JOBSTART, 10.1 
percent of all experimentals compared to 12.6 percent of all controls were arrested, for a statistically 
significant difference of 2.6 percentage points. A larger impact was observed for men without a prior 
arrest; 11.6 percent of experimentals reported an arrest versus 17.6 percent of controls, for a 6.4 
percentage point difference. However, for both the full sample and men who had been arrested 
between age 16 and entry into JOBSTART without a prior arrest, there was only a small difference 
in arrests during the entire four-year period, implying that involvement in the program made a 
difference that did not continue once participation ended. The level of involvement in the criminal 
justice system for all men in the sample — even though lower than for those with a prior arrest — 
is striking; 47 percent of both experimentals and controls were arrested at least once during the four 
years of follow-up. 


¢ JOBSTART also led to a reduction in the use of drugs other than marijuana for 
the full sampie during the fourth year of follow-up. 
In the four-year survey, respondents were asked whether they had used various drugs during 
the previous year, and experimentals reported significantly lower use of drugs other than marijuana 
compared to controls (4.1 percent versus 5.8 percent). 


Earnings Impacts by Site 


There is strong policy interest in the influence of site characteristics on impacts, but it is 
difficult to draw clear lessons from the JOBSTART Demonstration. This study was not designed to 
address rigorously how variations in the way the basic JOBSTART model was implemented or 
differences in local conditions niay have affected impacts. Youths were randomly assigned to the 
experimental or control group in each of the 13 sites, rather than to different types of programs or 
different labor markets, which would be necessary for an experimental test of the influence of 
alternative approaches or local environments on program effectiveness. 

Attempts to use differences in impacts across sites to understand the influence of program 
characteristics face serious problems. There were only 13 sites in the study; the average sample at 
each site was small (about 150) so that most site-level impacts were not statistically significant; and 
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the differences among the sites in impacts on total four-year earnings — a key outcome — were also 
not statistically significant. Finally, the sites differed in many characteristics including the 
backgrounds of the youths in the sample, labor market conditions, the level of alternative services 
received by controls, and many dimensions of their JOBSTART programs, making it difficult to 
isolate the factors contributing to impacts. 

Despite these difficulties in analyzing site-level impacts, one site did stand out from the others. 

* The Center for Employment Training (CET) in San Jose, California, had 

earnings impacts of more than $6,000 for the third and fourth years combined, 
and more than $6,700 for the entire four-year period. These impacts were 
statistically significant and substantially larger than those at any other site. 

Although site-level findings from JOBSTART can only be suggestive, these strong impacts at 
CET/San Jose, coupled with similar findings in the Minority Female Single Parent (MFSP) 
Demonstration, provide growing evidence of the effectiveness of CET/San Jose’s program. 

The reasons for these strong impacts remain uncertain because CET/San Jose is an unusual 
program in many respects. It is known for providing highly integrated education and training, but in 
JOBSTART this did not appear to be a fact: r necessarily leading to strong site impacts; the other 
JOBSTART site with integrated education and training had negative four-year earnings impacts. 
Other unique features of CET/San Jose’s JOBSTART program included a clear organizational focus 
on employment as the program goal, little upfront screening of applicants, training in occupations in 
demand in the labor market, relatively intensive services concentrated during a short period of time, 
strong job placement efforts, and a high-wage labor market. 

Overall, there was no clear pattern of impacts among the 13 sites that supports more general 
conclusions sbout the effect of program characteristics on site impacts. One aspect of program design 
that was of special interest in the study was the choice of offering education followed by occupational 
training (sequential programs) versus offering education and training simultaneously (concurrent 
programs). When the JOBSTART sites are divided into two groups based on this program feature, 
within each group there are sites with positive and negative earnings impacts. 


Findings on the Benefits and Costs of JOBSTART 


The benefit-cost analysis examines how overall program benefits compare to overall program 
costs from three perspectives. In doing so, the benefit-cost framework summarizes many different 
effects that can be attributed to JOBSTART. From the perspective of the youths given access to the 
program, the analysis examines whether the program produced greater benefits than costs during the 
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four years of follow-up and, if so, when during the follow-up period such a payoff began to occur. 
It also summarizes the program's total net costs and benefits to program funders (labeled “taxpayers,” 
as would be the case should JOBSTART become a large-scale program) and to society as a whole. 

Although it is not possible to determine a dollar value for all the costs and benefits of 
JOBSTART, most key items in the analysis — such as program costs, earnings increases, and changes 
in public assistance receipt — are quantifiable. The analytical framework allows for a compariscn of 
the impacts produced by the increased services received by the experimental group to the net costs 
of providing these services. In the analysis, benefits are not projected beyond the four-year follow-up 
period, but there is a discussion of the likelihood that longer follow-up would change the results. 

* The costs of providing JOBSTART services were approximately $4,500 per 

experimental. 

The benefit-cost analysis takes into account the costs of all program activities that were related 
to the operation of JOBSTART. Included were the costs of participation in the education and 
training components of JOBSTART, as well as the costs of services received through other programs. 
These JOBSTART costs were $4,548 per experimental. About 85 percent of this amount financed 
the education, training, and job placement activities, with the remainder used for support services. 

© The net costs of the additional services received by experimentals over controls 

were also approximately $4,500 per experimental. 

The program’s net costs reflect the incremental use of education and training resources by the 
experimental group over the amount used by controls. In the JOBSTART Demonstration, 
experimentals and controls received approximately equal amounts of non- JOBSTART services over 
the four years of follow-up. Because the non-JOBSTART costs for experimentals and controls are 
virtually the same, they cancel each other out; therefore, the net costs of providing education and 
training services to the experimentals were simply the costs of providing JOBSTART services. 

© At the end of the four-year follow-up period, JOBSTART had begun to pay off 

for participants. 

JOBSTART experimentals experienced a net gain in income of $141 per person over the four- 
year follow-up period. This small gain is the result of substantial losses in the first two years of the 
study, followed by a steady payoff in the last two years. Behind these changes in overall income are 
effects on earnings, fringe benefits, and medical coverage, as well as generally small reductions in the 
receipt of various forms of public assistance. 
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¢ Women who were custodial mothers at entry into JOBSTART experienced an 

estimated $1,004 increase in net income. 

In terms of their family income, women who were custodial mothers at program entry benefited 
most from their JOBSTART experience. For them, the program produced earnings gains as well as 
increases in public assistance payments. However, much of these increases in public assistance 
appears to have resulted from the increased birth rate for this subgroup, implying that the higher 
income for experimentals had to support somewhat larger households than for controls. 

¢ For other women and for men, the effect of JOBSTART on their income 

remained negative after four years of follow-up. Program payoffs for these two 
groups were insufficient to offset the opportunity cost of participation or the 
loss of public assistance. 

Women who were not custodial mothers at entry into the program experienced positive overall 
impacts on earnings, despite substantial opportunity costs in the first year of the study. However, a 
subsequent decline in public benefits (in part owing to a slightly reduced birth rate compared to the 
control group) exceeded these small earnings gains, resulting in an overall loss for these women. For 
men, earnings gains in the third and fourth years of follow-up could not offset the large initial losses 
experienced by this group. 

For both of these subgroups, the all losses occurred in spite of consistent improvement in 
the benefit-cost picture throughout the follow-up period. Thus, if observed trends in impacts 
continue, the net program impact on the income of experimentals in these groups may become 
positive as well. 

¢ For taxpayers, the resources devoted to funding JOBSTART were substantial, 

while tax payments by participants increased only slightly and savings in public 
services were modest or nonexistent. 

From the perspective of taxpayers, the key comparison is the cost of program services versus 
increased government revenue (through greater tax payments by participants who earn more) and 
reductions in public spending. The net costs of JOBSTART services were approximately $4,500 per 
experimental. Compared to these costs, the quantified benefits to taxpayers at the four-year point 
were relatively small. For the full sample, the small earnings impact generated only a slight increase 
in tax payments, and spending on key public assistance programs changed only slightly. There was 
a $74 increase in the four-year net present value of AFDC payments and a $28 increase in General 
Assistance. For Food Stamps there was a $34 reduction. The overall conclusions are similar for the 
three key subgroups of men, custodial mothers, and all other women, although those for the other 
women are somewhat less negative owing to substantial savings in AFDC payments for this subgroup. 
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¢ For society, the resources devoted to JOBSTART exceeded the benefits produced 
by the program. 
The small overall gains for participants included in this limited benefit-cost analysis were not 
nearly large enough to offset the program's overall cost. Therefore, within the confines of this 
analysis, from a societal perspective the program was not cost-effective. 


This assessment, however, is based on the assumption that a dollar lost to taxpayers is equally 
valuable to socie,y as a dollar gained by program participants: that is, there is no public value in 
redistributing income for disadvantaged youths through employment-oriented programs. If policy- 


makers do find independent value in such an approach, this could change the conclusions of the 
analysis. 

These results could also change in other ways. First, the earnings gains observed in years three 
and four of the follow-up period may grow — rather than remain steady or decline — over time. 
However, without longer follow-up it is impossible to know whether this will occur. Second, it is 
possible that in a more detailed benefit-cost analysis, the observed impacts on criminal arrests and 
drug use (especially for men with a prior arrest) may have substantial monetary value to taxpayers. 
Finally, other intangible benefits produced by the program (for example, the public benefits from a 
more educated citizenry) may be deemed substantial. 


The JOBSTART findings — coupled with those from other studies — provide mounting 
evidence of the challenge of serving disadvantaged out-of-school youths. While the message from 
recent research is far from optimistic, the conclusion that no program has enduring effects is overly 
pessimistic. In JOBSTART, there were apparent earnings gains in the third and fourth years of 
follow-up, well after the end of program services. The pattern of earnings impacts observed for 
JOBSTART was similar to that expected for such a program: an initial period when participants 
forego earnings (the "opportunity costs” of being in the program), a succeeding period when they 
catch up with controls’ earnings, and finally a period when their earnings exceed those of controls. 

The central problem in JOBSTART concerned the magnitude and duration of these negative 
and positive earnings impacts, not the lack of any payoff or the decay of program impacts: The initial 
losses for some subgroups were too large and the later payoffs too modest, at least during the four- 
year follow-up period for this study. 

The final chapter of the full report — summarized briefly here — presents ideas based on 
JOBSTART and other research findings, and on operational experience, about possible ways to 
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change youth employment programs to improve earnings impacts. The difficulty in applying the 
JOBSTART impact findings to program implementation and operations is that trade-offs are 


inevitable. For example, most ways of responding to the problem of initial earnings losses (by 


providing jobs or income support to the youths) cost money and — unless later impacts substantially 
improve — worsen the benefit-cost picture. The goal of discussing various options is to help program 
designers and operators fashion a combination of responses to better target program recruitment, 
encourage more substantial participation, lessen the initial earnings losses (the opportunity cost of 
participation), and increase the earnings payoff in the later years. 


¢ Programs such as JOBSTART are likely to have their best effect if a substantial 
percentage of participants have serious barriers to employment. 


The relatively high employment rates for youths with fewer barriers to employment are likely 
to produce large initial earnings losses for such participants that will be hard to compensate for later. 
JOBSTART earnings impacts were strongest for those less likely to be employed in the absence of 
the program: men with a prior arrest and youths who left school for educational reasons rather than 
to take a job. Also, earnings impacts were better in absolute terms for women than for men over the 
entire four years, and in terms of the percentage change in the last two years. These findings 
illustrate the importance of including youths with substantial employment barriers in programs such 
as JOBSTART. 

Two cautions are necessary, however, in interpreting these findings. First, programs less 
intensive than JOBSTART may have a different pattern of impacts because the services provided are 
not able to help the youths overcome the employment problems they face. Second, the nature of the 
program experience can change if all — rather than some — participants face very serious barriers 
to success; there will be fewer role models and success stories to help motivate youths and provide 
satisfaction for staff. As the percentage of harder-to-serve clients increases, program managers should 
more closely monitor the achievement of intermediate and long-term milestones by participants, as 
well as the morale and motivation of the young people and staff. 

© Nonexperimental research using the JOBSTART sample suggests that there may 

be a threshold level of service receipt necessary before substantial earnings 
impacts emerge, so there is a case for continued efforts to increase participation 
hours and improve program retention. 

Given the wide range of hours of participation among experimentals in JOBSTART, questions 
remain about how the program "worked" (that is, what its impacts were) for those who participated 
intensively. To answer them involves estimating impacts for subgroups defined by post-random 
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assignment behavior, which poses special analytical problems and makes any conclusions especially 
tentative. Nevertheless, a nonexperimental analysis conducted by MDRC suggests that post-program 
earnings impacts were negative for youths in the bottom third of JOBSTART participation hours, 
modest (and about equal to the experimental analysis's earnings impact estimate for the full 48-month 
sample) for those in the middle third, and very large for those in the top third. Although these 
results are not definitive, they suggest that continued efforts to improve youths’ program participation 
are important. Analysis of this and related issues will be pursued further in a forthcoming technical 
paper. 

Several types of operational responses to these findings are possible. First, there were some 
obvious problems at certain sites in the way JOBSTART was structured that inhibited participation. 
At sites where one agency first provided education and another agency subsequently provided training 
(sequential/brokered sites), most of the youths never made the transition to training, which seriously 
lowered participation. Efforts should be made at such sites to develop agreements with training 
providers to give referrals from the education agency priority for admission to training; to provide 
opportunities for participants to explore training options during education, or even start training prior 
to the completion of education; to schedule the transition from education to training to avoid time 
lags; to streamline the application process at the training agency; and to provide a case manager at 
the education agency to facilitate and monitor the transition of young people into training. 

Second, as discussed below, participation could be enhanced by providing the young people 
with financial support (preferably through paid work experience) and by helping them address the 
emotional and social problems they face in their daily lives. 

¢ Efforts should be made to reduce the opportunity costs youths must pay to 

participate in programs such as JOBSTART. 

There are several ways program operators and policymakers might accomplish this objective. 
One approach worth further exploration is to provide participants with income by helping them 
secure and keep training-related part-time work (scheduling program activities to allow program 
participation and work to be combined). This approach, which might increase program retention, has 
the added benefit of giving young people an opportunity to apply what they learn in the classroom 
to actual work situations, thereby adding relevance and meaning to their training experience. It also 
offers a way for those with little exposure to the world of work to become acclimated to the work 
environment and learn more about the importance of punctuality, relationships with supervisors, and 
other basic aspects of the workplace. 
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It is true that past research on work experience as a sole program service for young people 
found little effect on long-term employment and earnings. But the experience of JOBSTART and 
other recent youth programs suggests that work experience, when provided in combination with skill- 
building services, has the promise to improve long-term impacts. 

¢ The psychological and developmental needs of many seriously disadvantaged 

young people must be addressed or these youths will be unable to benefit from 
employment and training programs. 

Many of the young people in employment and training programs have lived in relative poverty 
and isolation for much of their lives. They have not had the adult support necessary to reach the 
point in their development where they are ready for the responsibilities of adulthood. Many are not 
even able to take advantage of skill-building opportunities such as JOBSTART. In the 
demonstration, staff in many sites gradually recognized the importance of addressing these issues 
directly and expanded the scope of their work, moving beyond narrowly defined employment issues 
to help youths address problems of emotional development, personal safety, housing, health care, and 
interpersonal skills. Some innovative programs (such as YouthBuild and state and local youth 
conservation corps programs) have coupled education and training with opportunities to enhance 
young people’s self-image and to develop interpersonal and leadership skills; these opportunities often 
include paid work experience providing services of value to the youths’ community. However, much 
is yet to be learned about the best ways to address young people’s psychological and developmental 
needs within JTPA, the nation’s major employment and training program, and further 
experimentation and innovation are needed. 


* The JOBSTART findings and other research and operational experience 
reinforce the importance of strengthening the link between program training 


and services and the job market. 

Strengthening the link between education and training and the job market through greater 
employer involvement is key to the success of programs and their participants. Through such 
involvement, programs can determine the employment needs of the local job market and train youths 
to meet them. In addition, employers can provide participants with opportunities to understand more 
clearly how their training applies to the workplace. Early links with employers also increase the 
possibility that they will hire participants after the program. 

To create strong program/employer relationships and to place disadvantaged youths in jobs 
requires experienced specialists in job placement who see serving these young people — who may 
remain difficult to place even after program participation — as their central mission. If placement 
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assistance is left to larger organizations such as the state employment service, which have a broad 
range of clients, there is the risk that hard-to-serve youths such as those in JOBSTART will "fall 
through the cracks." Youth employment programs would often be more effective if they developed 
strong job placement services. 

© Longer-term assistance after placement in a job also appears to be a promising 

innovation. 

Experience in JOBSTART and other youth employment programs strongly suggests that the 
battle is not won when a young person is placed in a job. For many youths, there is a need for 
continued help to adjust to the demands of supervisors and the workplace, while for others new issues 


of child care or transportation emerge. Further, since the initial jobs of young school dropouts rarely 


pay well, there is a need for staff to help youths make a favorable transition from a first job to a 
better, second job, or to further training and education. With the recent amendments to JTPA 
authorizing services for up to a year after initial job placement, such longer-term assistance will now 
be easier to provide. 


CHAPTER 1 
GINS AND G O 0 


The past two decades have been hard years for young Americans with limited job skills. The 
changing structure of the U.S. economy has meant declining opportunities to enter higher-paying 
careers in manufacturing, while at the same time well-paying jobs in the growing service sector have 
been demanding ever-higher education levels, and the inflation-adjusted earnings of young school 
dropouts have starkly declined. Although most young people between the ages of 16 and 24 were 
able to make the transition from adolescent to employed, self-sufficient worker, many of those 
without a strong education increasingly found this transition difficult. 

Overwhelmingly, the problem of youth unemployment in recent years has been concentrated 
among high school dropouts who come from poor families. Many are members of minority groups, 
some of whom confront the continuing existence of job discrimination.! A further salient aspect of 
the problem is the growing proportion of jobless young men who are not in the labor force — that 
is, no longer looking for work. In 1970, among all young men ages 16 to 24 who had no high school 
credential and who were not enrolled in school, only 13 percent were defined as not in the labor 
force; by 1991 this proportion had risen to 25 percent. Among black males the comparable figures 
were 40 percent in 1970 and 55 percent in 1991.2 

For a time there was a sense of optimism that demographic changes would counteract, at least 
in part, these economic trends. The relative shortage of young, entry-level workers arising because 
of the "baby bust" of the late 1960s and 1970s was expected to force employers to recruit and train 
individuals they might otherwise not find attractive, pulling young school dropouts into jobs. 

Unfortunately, that did not happen. As shown in Figure 1.1, the employment rate (those with 
any paid work as a percentage of all people in a group) of all young school dropouts in the United 
States has not increased over the last two decades, and the rates for males and blacks have actually 
declined, with a sharp drop in the recession that began in the late 1980s. The employment rate of 
females has been relatively stable, but low (about 30 percent), and it also declined sharply at the end 
of the 1980s. 


ISee Kirschenman and Neckerman, 1991, for a recent discussion of the ways in which racial discrimination 
affects hiring decisions. 
2U,.S. Department of Labor, Bureau of Labor Statistics, 1992. 
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FIGURE 1.1 


EMPLOYMENT RATES OF 16- TO 24-YEAR-OLDS WHO DO NOT HAVE A 
HIGH SCHOOL DIPLOMA OR GED AND WHO ARE NOT ENROLLED IN SCHOOL, BY SUBGROUP 


” All Subgroups 
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Year 


SOURCE: U.S. Department of Labor, Bureau of Labor Statistics, 1969, 1992. 


Further, among those young dropouts who worked, earnings (adjusted for inflation) dropped 
dramatically, by more than 40 percent between the mid-1970s and 1980s. As Freeman and Holzer 
have noted, “Declines in real earnings and in employment rates of the magnitudes observed are 
historically unprecedented. The problem facing the young less educated workers in the United States 
is not one of youth per se, but of change in the job market for persons of their skills"? As Holzer 
observed in another paper: 

The shift of employment away from manufacturing toward the trade and service sectors, 

and especially the decline of operative and laborer jobs within the manufacturing sector, 

. . . Was particularly strong in the 1980s. These trends have eliminated many jobs in which 

less-skilled workers could earn fairly high wages. . .. The evidence on current employer 

perceptions and hiring practices, including growing employer emphasis on certain types 

of skills, and the current weakness in literacy levels for dropouts and minorities all suggest 

that any current skill imbalances will continue or worsen in the 1990s. 

The negative consequences of these trends extend well beyond the lives of the young people 
themselves, affecting both the general public and the business community. There is strong evidence 
that the incidence of poverty, welfare receipt, criminal activity, and unwed parenthood is significantly 
higher for those with poor basic skills than for the population as a whole (Berlin and Sum, 1988; 
Wilson, 1987). Society bears the cost in the form of social disruption and the need for increased 
public services. 

Together these developments have prompted renewed interest in solving the problems that low- 
skilled young people have in making the transition to the working world. Commissions appointed by 
the secretaries of several federal agencies and by prominent foundations have called for renewed 
investment in education and training to upgrade the skills of entrants into the labor force.’ The 
Family Support Act of 1988 (FSA) called for increased educational services for young women 
receiving Aid to Families with Dependent Children (AFDC); and the 1992 amendments to the Job 
Training Partnership Act (JTPA) created a separate year-round youth program, requiring that local 
JTPA programs target services to youths with serious barriers to employment and provide increased 
educational services for young school dropouts. In his election campaign, President Bill Clinton 
endorsed a plan to create a national network of youth opportunity centers to provide young school 
dropouts with an alternative way to learn the skills they need to succeed in the workplace. It is likely 


3Freeman and Holzer, 1991, pp. 5-6. 

‘Holzer, 1992, pp. 252-60. 

SSee, for example, Johnson and Packer, 1987; U.S. Department of Labor, Secretary’s Commission on 
Achieving Necessary Skills, 1991; William T. Grant Commission on Work, Family and Citizenship, 1988; and 
National Center on Education and the Economy, 1990. 
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that legislation calling for combined education, training, and other services for low-skilled youths will 
soon be introduced in Congress. 


I. The JOBSTART Demonstration 


The JOBSTART Demonstration tested this type of program for young school dropouts; thus, 
it provides evidence of the promise of the approach and of the strengths of different implementation 
practices, and offers lessons about the limitations of these varying practices when operated within the 
current programmatic context. The key features of the JOBSTART progratn are summarized in 
Table 1.1.° Issues of particular importance in the demonstration were the target population and the 
services available to participants. The 13 study sites recruited and enrolled young, primarily black and 
Hispanic high school dropouts who read below the eighth-grade level when they entered the program. 
This group had been underserved in the early years of JTPA, since many local programs enrolled 
people with higher reading levels who needed less intensive services. JOBSTART combined basic 
education, training in occupational skills, limited support services (primarily assistance with child care 
and transportation), and job placement assistance. Operating support consisted primarily of funds 
provided under JTPA, the nation’s largest funder of employment and training programs for 
economically disadvantaged people.’ Typically, JTPA services provided to participants — even to 
young school dropouts — have been less intensive than those in JOBSTART. JOBSTART was a test 
of what happens when JTPA is pushed to serve an educationally and economically disadvantaged 
group in a relatively intensive way. 

Developed and overseen by the Manpower Demonstration Research Corporation (MDRC), 
JOBSTART was implemented in 13 study sites: four adult schools (three adult vocational schools and 
one community college); six community-based organizations (CBOs); and three nonresidential Job 
Corps programs. The demonstration ran from 1985 to 1989, before recent amendments to JTPA that 
make the program more conducive to initiatives such as JOBSTART. 

The goals of JOBSTART were ambitious: 


¢ The sites were to recruit young, economically disadvantaged high school dropouts 
with low basic skills. 


SJOBSTART is described in more detail later in this chapter and in Chapter 3. 

TJOBSTART funds came from Title ILA of JTPA, the largest part of the JTPA program, or Title IVB, 
which funds the Job Corps Centers. Additional, special funding, provided primarily by foundations, was 
available to sites in the demonstration to cover a portion of the expenses of participating in the research. 
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TABLE 1.1 


THE JOBSTART PROGRAM GUIDELINES 


Target Population 10 be eligible for JOBSTART, individuals had to be: 
17 to 21 years old 
¢ school dropouts without a high school diploma or GED 
° reading below the eighth-grade level on a standardized 
test 
* economically disadvantaged” 


Basic Education shes were to implement a curriculum that: 
was self-paced and competency-based 
* was computer-managed and -assisted, if possible 
* was a minimum of 200 hours in length 
¢ focused on reading, communication, and basic 
computation skills 


Occupational Skills Training oe were to implement a curriculum that: 

was in a classroom setting 

* combined theory and hands-on experience 

* prepared enrollees for jobs in high-demand occupations 

* provided at least 500 hours of training 

e had been developed with the assistance of the private 
sector to ensure that graduates would meet the entry- 
level requirements of local employers 


Training-Related Support Services Services were to be tailored to individual needs and were to 

include, in addition to transportation and child care, some 

Comnbiiation of Ue tonawing: 
work-readiness and life skilis training 
personal and vocational counseling, mentoring, tutorial 
assistance, and referral to external support systems 

* needs-based payments or incentive payments tied to 
length of stay, program attendance, or performance 


Job Development and Placement JOBSTART operators and/or their subcontractors were to be 
Assistance responsible for assisting participants in finding training- 
related jobs 


SOURCE: Manpower Demonstration Research Corporation, 1985. 
NOTES: °To help meet enrollment targets, each site was allowed to enroll individuals — up to 
economically 
person must be receiving public assistance; have family income at or below the poverty line or 70 
percent of the lowest living standard income level; be homeless, under the definition of federal 


Statutes; or, in some cases, be a handicapped adult whose own income fits within the guidelines but 
whose family income exceeds it. 


aGin 


* They were to put in place a package of services that would address the needs of 
the youths. 


* The young recruits would have to take advantage of this opportunity and invest 
their time and effort in education and training activities. 


* Their efforts were expected to yield them new skills, as measured, for example, by 
their completing high school or passing the General Educational Development 
(GED) test for high school equivalency certification.® 


° Over time, those new skills were expected to translate into greater employment 
and earnings than the youths otherwise would have had, and into less need to rely 
on public assistance. 

Understanding whether these goals were met is the purpose of the JOBSTART evaluation. 

Earlier reports on the JOBSTART Demonstration, summarized and updated in this document, 
found that the first four conditions for program impacts listed above were generally met: The program 
sites were able to recruit the target group of youths; they generally offered the intended services; 
many youths participated in education and training; and their participation led to increased GED 
receipt.!° In sum, the JOBSTART program model received a “fair test” in the demonstration. 
Thus, it presents a unique opportunity to see whether and how employers responded once the 
disadvantaged young people had increased their educational attainment and vocational skills. 

This final JOBSTART report addresses the fifth goal: whether helping young, disadvantaged 
school dropouts increase their educational attainment leads to increased earnings, especially in the 
short run. This question has not been answered by previous research. Numerous studies have found 
that people with higher levels of education earn more than those with no high school diploma or 
lower educational levels.'! But direct rigorous tests of the impact of increasing the educational 
attainment of a group of disadvantaged young people who did not complete high school are rare. 


®TMhe GED test is a national examination produced and administered by the GED Testing Service of the 
American Council on Education in Washington, D.C. Individual states have different criteria governing who 
may take the examination, different standards for passing scores, and different credentials awarded to those 
passing (for example, a state high school equivalency certificate or a state high school diploma). In accordance 
with common usage, the credential is referred to in this report as a GED certificate or, simply, a GED. 
*The program could also affect other aspects of the young people's lives. They might be better able to 
live on their own instead of with their parents, more likely to postpone childbearing (because they see 
Opporiunniet in the labor market), or less likely to engage in criminal behavior. 
1See Auspos, 1987; Auspos et al., 1989; and Cave and Doolittle, 1991. 
1For example, Berlin and Sum, 1988, found that among young men and women during the late 1970s, 
the payoff of an additional year of secondary school was approximately $700 in incrcased annual income, while 
a high school diploma had a “credential effect" of about $925 per year. The authors also attempted to control 
for the level of basic skills of individuals by including youths’ scores on the Armed Forces Qualifying Test as 
an independent variable in a regression. The estimated effect on annual carnings of an additional grade- 
equivalent of basic skills (for example, progressing from a seventh- to an eighth-grade reading level) was $185. 
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Although, in general, increasing the skills of young people does increase their earning capacity, 
there are two important countervailing effects of a program such as JOBSTART, at least in the short 
run. Participation in an intensive program can increase the academic and occupational skills of young 
people. At the same time, however, it pulls participants out of the labor force. Not only do they give 
up earnings while in the program, but they also have less chance to gain workplace skills and seniority 
through on-the-job experience. For young people, especially those with low levels of educational 
attainment, work experience is an important source of new skills and greater job stability and 
wages.!? The JOBSTART evaluation assesses which of these two effects prevails. 

This report covers four years in the lives of the young people in the demonstration. While this 
follow-up period is long for this type of evaluation, the JOBSTART youths were at most 25 years of 
age when this study's follow-up period ended. Thus, this final report cannot present the long-term 
impacts of the program, but it is nevertheless appropriate to take stock of the program's impacts, 
benefits, and costs at this point. One key question addressed is whether the young people who 
participated in JOBSTART were better off because of the program: In the post-program follow-up 
period, did they reap employment gains sufficient to compensate them for the time they committed 
to the program and their resulting foregone earnings? A second question concerns the value of 
JOBSTART from a social economic-efficiency perspective. Because the program involved a 
substantial initial investment of funds to provide services, unless a pattern of significant, positive 
program impacts appears during this four-year period, it is unlikely that the social benefits resulting 
from the program will exceed in value the resources devoted to it. A final question is whether the 
experience of the JOBSTART Demonstration provides lessons for the development and 
implementation of new programs for low-skilled school dropouts. 

All the key findings from the demonstration are presented in this report to assist the reader in 
understanding the JOBSTART story. The first four chapters, on the nature of the JOBSTART 
Demonstration and services, are modeled closely on material presented in prior reports, with findings 
updated to reflect both the longer follow-up data now available and the sample of individuals used 
for the four-year impact analysis.'> Succeeding chapters present new, four-year findings on the 


an unpublished analysis by Andrew Sum, using the Current Population Survey, found that the average 
earnings of high school dropouts increase noticeably with age during the late teenage years and early and mid- 
twenties. In 1986, for example, the average annual earnings of 19-year-olds were about $2,000 higher than 

those of 18-year-olds. 
13s discussed in Chapter 2, JOBSTART follow-up data were collected through surveys conducted one, 
two, and four years after a young person became part of the research sample. Because each survey “wave" 
(continued...) 


o7e 


o7 


employment and other effects of the program, and its benefits and costs from the perspectives of both 


participants and society as a whole. The analysis highlights throughout how the program worked 
differently for subgroups of youths that were defined based on their characteristics when entering the 
program: for example, gender, parental status, and ethnicity. The concluding chapter offers reflections 
on the lessons of the JOBSTART Demonstration for future employment and training policies and 
programs. 

Before presenting the findings, however, it is important to describe the changing research and 
policy context in which JOBSTART was developed, implemented, and evaluated. Key topics in this 
discussion are the continuing research on the effectiveness of various education and training 
interventions and the effect of JTPA rules on the design and operation of JOBSTART, including 
important legislative changes since JOBSTART began that signal greater attention to youth 
employment programs and more flexibility in responding to the needs of youths with severe barriers 
to employment. The JOBSTART Demonstration and program are then described in greater detail, 
and the chapter concludes with a brief overview of the JOBSTART evaluation and an outline of the 
remaining chapters in this report. 


The research findings on youth employment programs available in the mid-1980s, and the then 
current rules of JTPA — the major funder of JOBSTART operations — strongly influenced the 
development of the JOBSTART program model and its implementation. More recent research 
findings and changes in JTPA and other programs have been important in shaping the key questions 
addressed in the evaluation and in developing implications for future policy initiatives. 


1. Prior research and operational experience from programs for young school dropouts. 
At the time the JOBSTART Demonstration began, program designers secking insights from the 
previous research on programs serving young school dropouts found few solid success stories on which 
to base new efforts. Many types of programs had been tried, but nearly all the evaluations found 
unfavorable results, were inconclusive, or were seriously flawed.!4 


13(..continued) 
succeeded in collecting information from a slightly different subset of the entire research sample, the 
individuals included in cach report vary slightly. 
144 common methodological problem was the absence of an appropriate group (one that was similar to 
participants but not served by the program) against which the experiences of the group pa irs 
(con 
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The one influential exception to this pattern was the residential Job Corps, which a study found 


to be effective in increasing the educational attainment and earnings of young dropouts.!5 The 
residential Job Corps provides basic skills education, occupational training, life skills instruction, work 
experience, job placement assistance, health care, counseling, and other support services to youths 
who live at centers (often outside urban areas) and participate in the program for up to two years. 
About 80 percent of Job Corps participants have not completed high school. 

As would be expected in an intensive program of skills enhancement, Job Corps participants 
initially earned less than their comparison group counterparts, and this “reduction” in earnings lasted 
until about six months after their participation in the program ended.” This delay in the payoff 
of the program occurred because those in the comparison group had accrued more work experierce 
(an important source of skills-building), seniority and protection against layoffs, and promotions. !8 
Earnings gains began to appear about six months after participants left the program and continued 
throughout the remaining four years of follow-up. Not only did the research record find positive 
impacts for the residential Job Corps program, but it also found program benefits to be greater than 
program costs. Especially encouraging was the program's positive findings for young male dropouts, 
a group that had proven especially hard to serve in many previous programs. 


14(. continued) 
be compared. Without such a comparison, evaluators frequently confused outcomes that followed a program 
with the real difference a program made — in the language of evaluation, its "impacts." For example, the 
outcomes of a program might include a post-program job placement rate of 50 percent. However, the 
employment rate of the appropriate comparison group might also be 50 percent, suggesting that the program 
had no impact on employment rates. See Betsey et al., 1985, and the discussion in Chapter 2 of this report 
for more on this issue. 
1Ssee Mallar et al., 1982. The results of this study are not precisely comparable to those of JOBSTART 
since the two studies used different research methodologies. As discussed in Chapter 2, JOBSTART used a 
random assignment research design, while the Job Corps study used a comparison group design. Many of the 
findings in the Job Corps study are reported for periods beginning with the end of program participation, while 
JOBSTART started tracking sample members and calculating impacts at the point of random assignment. 
Further, the Job Corps sample of participants excluded those who were active in the program for a short 
period. Since those who stay in a program for only a short time are unlikely to benefit much from 
participation, their exclusion from the sample probably raised the average long-term impacts presented in the 
analysis for Job Corps participants. In the JOBSTART study, these “short stayers" ~ and in fact a small group 
who were randomly assigned to JOBSTART but never participated — were part of the sample. 
'6Some Job Corps Centers also operate a nonresidential program. These were not included in this earlier 
study. As previously noted, three nonresidential Job Corps programs were included in the JOBSTART 
Demonstration. 
17This would occur because of participants’ foregone earnings and lost opportunities for on-the-job skills 
enhancement while they were in the program, as discussed earlier in this chapter. 
1Evaluations of the Job Corps discussed the problem of the post-program transition back into the labor 
market that led to these initial negative impacts. See Mallar et al., 1978, 1980. 
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The residential Job Corps, however, could not be offered to all dropouts: It is a relatively 
expensive program (averaging about $5,000 per person per year in 1977, when the study was done, 
and about $15,000 currently), is accessible only to those willing and able to live away from home, 
requires development of work experience positions with employers, and is clearly not the answer for 
all disadvantaged youths. 

Other efforts to directly connect young people with work — either by helping them look for 
work more effectively or providing subsidized work experience — were tested in demonstrations in 
the early 1980s. An evaluation of job search assistance for youths found that the program produced 
short-term increases in employment and earnings, but that in the long run participants were no better 
off than a comparison group.'? As for the most common youth employment strategy —. subsidized 
work experience — two evaluations failed to find any long-term impacts on educational attainment, 
employment, or earnings for young dropouts.” 

Thus, the research record of the mid-1980s put the Job Corps in a special category as an 
effective program for raising the employment and earnings of young school dropouts.2! Among the 
questions left open by the existing research, however, was whether the Job Corps approach could be 
successfully adapted to a new setting: a nonresidential program operated by other agencies not able 
to offer comprehensive support services and without the Job Corps’ staff training, facilities, and 
curricula. A shift to a nonresidential program — as represented by the JOBSTART Demonstration 
— is an important one, since some of the benefits of the traditional Job Corps program seemed to 


19The demonstration assessed the effectiveness of a program providing job search assistance through 
simulated interviews, seminars on job-seeking techniques, and help in making contact with potential employers. 
See Public/Private Ventures, 1983. Its finding differs from that of research on job search assistance programs 
for women receiving AFDC, which did find long-term employment and earnings impacts. See Gueron and 
Pauly, 1991. 

National Supported Work Demonstration, managed by MDRC in the late 1970s, used a random 
assignment research design and enrolled very disadvantaged young dropouts (many with a criminal record) in 
a 12- to 18-month program of paid work experience with gradually increasing job responsibilities. Program 
impacts for this group were not positive, even though the program proved successful for long-term welfare 
recipients. See Maynard, 1980. The Youth Incentive Entitlement Pilot Projects (YIEPP), which offered 
subsidized minimum-wage jobs to high school students and dropouts who returned to school, also was 
ineffective for dropouts. See Gueron, 1984. While the program did increase the employment and earnings 
of young people still in school, evaluators found that the offer did not induce dropouts to return to and remain 
in regular high school. Many of those who did return dropped out a second time, and there were no effects 
on educational attainment, employment, or earnings for dropouts. 

2!The National Academy of Sciences, in its review of research on employment programs for young people, 
pointed out the distinction between the failure of research to provide adequate evidence of program 
effectiveness and the finding that a program is ineffective. (Betsey et al., 1985.) 


-10- 


60 


result from its residential nature.22 While the residential nature of the program may have been a 
factor in its success, it did pose problems for some young people. For young mothers with child care 
responsibilities, the program demanded too much time away from home, and it did not prove effective 
for them. Also, many young men and women did not wish to leave their communities. 

In 1983, the National Academy of Sciences convened a panel of experts on youth programs. 
Their assessment — summing up research findings — recommended further testing of the Job Corps 
approach in a nonresidential setting using random assignment to produce the most reliable 
findings. The JOBSTART Demonstration was, in part, a response to this call.”4 

In addition to this research record, program operators serving young, disadvantaged dropouts 
had identified a number of lessons that also informed the development of the JOBSTART 


Demonstration. When serving these youths, who often look back on past educational experiences 


with dissatisfaction, programs have to actively seek out participants rather than passively wait for 
volunteers to come forward. Program operators have also learned that achieving continued 
participation is not easy: Counseling and peer support have often proved useful in improving young 
people’s self-esteem and motivation, but even with these efforts, participation levels can be less than 
expected. 

Finally, program operators have increasingly become sensitive to the multiple needs of clients. 
For some economically disadvantaged young people, a low level of basic skills prevents them from 
taking advantage of occupational training. In addition, as is the case for many young people, 
economically disadvantaged youths may not be experienced in setting goals, making plans to achieve 
them, and following through with effective action. And finally, many young people have a pressing 


22For example, in a residential program, it is much easier to provide an intensive program of support 
services (including counseling outside class time, positive peer support, recreational activities, and health care) 
than when young people are active in the program for at most eight hours a day. Furthermore, the decline 
in criminal activity and substance abuse observed for Job Corps participants (especially during program 
participation) was partly attributable to their isolation in residential centers outside urban areas, or at least 
outside their previous neighborhood. 

assignment is generally recognized to be a reliable method of measuring the effectiveness of 
new employment and training programs. As discussed later in this report, it was used in the JOBSTART 
Demonstration. 

As the later discussion of the JOBSTART program model and its implementation will make clear, 
JOBSTART did not offer the same comprehensive list of support services available in the Job Corps. Nor 
did it use the same curricula in education or training, except in the three sites that already operated a 
nonresidential Job Corps program. Nationally, about 10 percent of Job Corps participants are in 
nonresidential 

of these lessons are summarized in 70001 Training and Employment Institute, 1988, and 
Public/Private Ventures, 1990. 
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need for immediate income, for themselves or their family, so programs must help them find a means 
of financial support while they invest in their future by enhancing their skills. 

2. The initial programmatic context of the JOBSTART Demonstration. Because the 
JOBSTART Demonstration did not have special program funding to support site operations, it was 
shaped in important ways by the need to find funding for local JOBSTART sites from existing 
programs. Since JTPA was the most likely source of local operational funds, its provisions and the 
local interpretation of them were central to the structure of the project. 

For a decade, the Job Training Partnership Act (JTPA) of 1982 has been the federal 
government's major program for funding employment and training for economically disadvantaged 
adults and youths. JTPA distributes the majority of its funds to states which, in turn, pass along most 
of what they get to local administrative entities called service delivery areas (SDAs).2° The federal 
JTPA statute sets general rules for program eligibility and allowable types of activities. An SDA's 
staff and private industry council (PIC) — often operating like a board of directors for the agency — 
determine what specific types of services are to be offered, which groups will get priority for services, 
and how service providers under contract to the SDA are to be evaluated and paid. 

The manner in which JTPA was initially implemented during the mid-1980s presented 
operational constraints that had to be taken into account in the design and implementation of the 
JOBSTART Demonstration: 
© Performence standards that made SDAs hesitant to serve youths with very poor 

skills. In designing and applying the performance measures used during the first 
five years of JTPA, federal, state, and local administrators focused on the 


proportion of participants placed in a job, their wages, and the cost per "success 
story."2” This encouraged SDAs and service providers to choose people who 


26Most JTPA funds under Title ILA, the largest part of JTPA, are distributed to states, using a formula 
based on the state’s number of unemployed and economically disadvantaged people. 

27From the early 1980s until program year 1987 (ending in June 1988), the performance of SDAs serving 
adults was judged by the following standards: the percentage of adults who found a job; the percentage of 
adults who were receiving welfare when they enrolled in JTPA and who found a job; the average wage at 
placement in a job; and the program cost per person entering employment. For youths, the standards included 
the percentage who found employment and the “positive termination rate,” defined as entering employment 
or other quantifiable measures of program success. These included attainment of employment competencies 
recognized by local private industry councils, completion of a level of schooling, enroliment in further non- 
Title IIA training, enlistment in the armed forces, return to school full-time, or (for 14- and 15-year-olds) 
completion of specified program objectives. The youth standards included the cost per “positive termination.” 
For each measure, the U.S. Department of Labor set national levels, which — at state option — could be 
adjusted to reflect the characteristics of those served and the conditions in the local labor market. 
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were more likely to achieve these successes at relatively modest costs.22 In 
seeking to serve school dropouts with poor skills in an intensive program, 
JOBSTART had to confront these issues. 


¢ Severe restrictions on paid work experience. Experience with public service 
employment under JTPA's predecessor, the Comprehensive Employment and 
Training Act (CETA), led Congress to eliminate most forms of paid work 
experience when enacting JTPA. Thus, it was difficult for JOBSTART to fund 
paid work experience, which had been an element of the residential Job Corps 
program; this led to JOBSTART as a test of education and classroom occupational 
training rather than education, training, and work experience. 


* Tight limitations on support services. The JTPA statute limited spending on 
support services (such as transportation and child care assistance) and needs-based 
cash payments, and completely eliminated the payment of stipends to participants. 
This increased the difficulty of retaining youths in the program.2? 


The early experience under JTPA prompted allegations that the program was making little real 
difference in participants’ lives because service providers tended to enroll more job-ready applicants 
(a practice known as "creaming"). Further, sharp declines in the unemployment rate during the 1980s, 
which allowed many more job-ready individuals to find work, caused a rethinking about whether 
JTPA should continue to emphasize quick placement of participants in a job. 

Over time, Congress, the U.S. Department of Labor (DOL), and program operators have all 
expressed renewed interest in intensive programs of education and training targeted on more 
disadvantaged youths. Responding to the early pattern of program operation, DOL changed its 
administrative practices and regulations and encouraged greater provision of intensive services for 
youths, easing the implementation problems faced in the later stage of the demonstration.” Soon 


Data from the mid-1980s illustrate the effects of these program priorities. During program years 1984 
to 1986, when the JOBSTART Demonstration was beginning, young dropouts constituted only 11 percent of 
all Title ILA participants and 27 percent of all young participants. Among young dropouts served under Title 
IIA nationally in 1986, only 23 percent received basic education, a service likely to promote their long-term 
em ty but unlikely to lead to immediate placement in a job. 

In addition, some types of performance contracts inhibited combined education and training, especially 
when the services were provided by different agencies. Many SDAs wrote contracts with service providers that 
linked payment to the achievement of the measures in the performance standards. This made it more difficult 
for those service providers that wished to serve youths with poor skills to be paid, since many youths might 
not reach the required level of performance, and complicated the administration of local programs where 
different agencies provided the education and training, since the youths might not have reached any 

benchmarks when they moved from the first agency to the second. 

3%Ip late 1987, DOL stated that "more emphasis must be placed on intensive investments in youth within 
JTPA" and recommended that “a significant portion of youths who participate .. . should receive 
competency-based instruction in either basic education or occupational skills” (Federal Register, December 16, 
1987). 
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thereafter, amendments to the regulations (effective in program year 1988) encouraged states to 
choose as the key standard for youth programs one that includes measures of increased educational 
and skills competencies. This increased the opportunities to include young dropouts with poor skills 
in JTPA?! 


B. Recent Developments Affecting the Interpretation of JOBSTART Findings 


Since the mid- to late 1980s, when JOBSTART began, research on the employment problems 
of youths and program reform have continued, Developments in both areas have heightened interest 
in the JOBSTART Demonstration. 

1. Research findings. Since the start of the JOBSTART Demonstration, two additional 
studies of youth programs have produced findings. Initial, 18-month findings are now available from 
the National JTPA Study, an examination of program implementation and impacts in 16 local 
programs.°2 As shown in Table 1.2, this study found that, for young out-of-school women, program 
impacts on earnings were slightly negative (-§182) but not statistically significant over the 18 months 
after random assignment. For young m=n, earnings losses were substantial (-$854) over the 18-month 
period, though these losses were primarily concentrated among young males with an arrest prior to 
applying for JTPA. For all these groups, the time trend is not encouraging, though again the results 
for males are more troubling. For males, earnings losses were present in every quarter, whereas for 
females, small (but statistically insignificant) earnings gains appeared in the fourth and fifth follow-up 
quarters. Earnings impact estimates for a subgroup of youths recommended for classroom training 
in occupational skills (not shown in the table), forming the closest counterpart to the JOBSTART 
services examined in the study, were also calculated. For this subgroup the earnings losses for 
females were greater than for males (-$542 versus -§259). Though longer follow-up, available later 
in the project, will be the final word, these early findings on youths are sobering, especially in contrast 
to the positive — though modest — 18-month impacts for adult men and women in the JTPA study. 


311p addition, in 1989 DOL issued a request for proposals for the Youth Opportunities Unlimited (YOU) 
Demonstration. Sites applying to participate were required to operate one of three innovative programs: a 
work experience program modeled on Ventures in Community Improvement (VICI), which operated from 
1978 to 1980; an alternative high schoo! program modeled on High School Redirection in Brooklyn, New 
York; or a program of education and training modeled after JOBSTART. In 1990, seven sites were chosen 
for the three-year demonstration; some of them chose to operate a program modeled after JOBSTART. An 
implementation study is part of that demonstration, which is separate from the JOBSTART Demonstration 
reported on here. 

32This study is being conducted by Abt Associates Inc., ICF, MDRC, New York University, and National 
Opinion Research Center (NORC), under funding by the U.S. Department of Labor. See Bloom et al., 1993. 
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TABLE 1.2 


NATIONAL JTPA STUDY IMPACTS ON EARNINGS 
FOR FEMALE AND MALE OUT-OF-SCHOOL YOUTHS 


Female Youths 


Experimentals Controls Difference Experimentais 
Follow-Up Period ($) ($) (S) ($) 


1,213 
1,526 
1,652 
1,738 
1,879 
1,874 
9,882 
Sample size 1,814 835 1,436 


SOURCE: Bloom et al., 1993. 


Male Youths 


Controls 
($) 


1,412 
1,598 
1,803 
1,876 
1,984 
2,063 
10,736 


708 


NOTES: There may be slight discrepancies in reported sums and differences of these adjusted means because of rounding. 
A two-tailed t-test was applied to each difference between average experimental and control group outcomes. Statistical 


significance levels are indicated as *** = 1 percent; ** = 5 percent; * = 10 percent. 
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Indeed, the findings on youths fueled already active debate about reforms of JTPA-funded youth 
programs, and led to further calls for more intensive programs targeted on disadvantaged youths. 

The findings from the Summer Training and Education Progra:n (STEP) Demonstration have 
also contributed to the debate about youth programs, although they deal with young people still in 
school. This demonstration, organized by Public/Private Ventures, provided education and part-time 
employment to “at-risk” JTPA-eligible students in the summers after their sophomore and junior years 
of high school, as well as additional services during the school year. Despite early signs of positive 
in-program impacts on basic skills and knowledge of sexual risks and contraceptive techniques, the 
program had no significant longer-term post-program impacts on key outcomes such as educational 
attainment, employment, earnings, parenting, and welfare receipt.?> This, plus the JTPA findings, 
posed starkly the difficulty of affecting the long-term prospects of economically and educationally 
disadvantaged young people. 

A second line of recent research is relevant for the debate about education and training 
programs for young dropouts. Despite the general conclusion that further education increases 
earnings, little is known about how employers assess receipt of a GED, the primary education 
outcome in "second chance” programs such as JOBSTART. The assumption behind such programs 
is that employers will view a GED as evidence of increased skills, but little is known about whether 
this is the case, how long it takes for a GED to pay off, and how any such payoff might vary among 
subgroups of youths and types of occupations. Alternatively, and less optimistically, a GED may do 
little to counter the negative impression created by the fact that the young person did not finish high 
school. Some recent research suggests that this is a real possibility; one study comparing the earnings 
of school dropouts, GED recipients, and high school graduates found little difference between 
dropouts and GED recipients, and a clear difference between both those groups and high school 
graduates.*4 These findings would be consistent with the view that employers do not consider 
receipt of a GED as signifying higher skills levels than those of a typical dropout. Alternately, it 
could mean that employers rely on a high school diploma as a sign of persistence (rather than a 
required level of basic skills), and that receipt of a GED could not overcome the negative signal 
conveyed by dropping out of school.”° 


33See Grossman and Sipe, 1992. 
See Cameron and Heckman, 1991. 
35See Layard and Psacharopoulos, 1974. 
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Intense research effort has focused on this issue in recent months, and JOBSTART — even 


though it is a program providing more than just assistance in passing the GED — has much to 
contribute to the debate. Chapters 4 through 6 show that JOBSTART succeeded in increasing the 
percentage of youths who passed the GED and examine the payoff in the labor market. 

2. Programmatic changes. Continued debate over JTPA led to the passage of amendments 
in late 1992, which become effective in July 1993. These amendments created a separate year-round 
youth title (Title IIC), with a focus on improving the long-term prospects of young people, and 
requires that 65 percent of youths served in local JTPA programs must mect one of several specified 
barriers to employment: basic-skills deficient, pregnant or a parent, experiencing a disability, homeless 
or a runaway, or a legal offender. Fifty percent of young people served must be out of school, and 
school dropouts under the age of 18 must enroll in an educational program. The amendments eased 
restrictions on the use of paid work experience as a service, reopening the program option (education 
and work experience) foreclosed by the original JTPA rules. They also revised performance standards 
to abolish cost standards and to reward services provided to “hard-to-serve" individuals, a group likely 
to need more intensive services such as those provided in JOBSTART. Finally, they imposed 
restrictions on the use of performance-based contracts, a form of service procurement that had been 
alleged to inhibit providing services to youths facing serious barriers to employment. While this 
increased interest in hard-to-serve youths came too late to affect the implementation of the 
JOBSTART Demonstration, it has heightened the importance of the project as an early test of a new 
direction for JTPA and has provided new flexibility in the design and implementation of programs 
for youths. 

Other policy developments also signal a growing emphasis on programs of education and 
employment services for low-skilled youths. The Job Opportunities and Basic Skills Training (JOBS) 
title of the Family Support Act of 1988 expands the obligations of AFDC mothers — and especially 
young mothers — to participate in activities intended to increase their employability, and of states and 
counties administering the AFDC program to offer more education and training than typically offered 
under the predecessor Work Incentive (WIN) Program.*© Expanded youth community service 
programs linked to education and training are also possible under other passed and proposed 


Especially relevant to JOBSTART is the fact that the JOBS legislation allows states to impose a 
participation obligation on AFDC parents under 20 years of age who lack a high school diploma or GED 
regardless of the age of their child. While single parents with children under age three are normally exempt 
from participation in JOBS programs, it is not the case for custodial parents under age 20 who have not 
graduated from high school or received a GED. For this group, education is normally presumed to be the 
appropriate first activity in JOBS. 
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legislation.*” Finally, many proposals to ease the transition from school to work for youths not 


attending college have been presented.°8 


Ill. it d the Guidelines 


MDRC began the JOBSTART Demonstration in 1985 with two purposes: (1) to determine the 
operational feasibility within JTPA of an intensive program incorporating several of the key elements 
of the residential Job Corps, and (2) to rigorously test its effectiveness.>? Local and state JTPA 
agencies provided most of the operational funding for the JOBSTART sites, but the MDRC 
evaluation was funded by an unusual consortium consisting of the U.S. Department of Labor, The 
Rockefeller Foundation, the Ford Foundation, Charles Stewart Mott Foundation, The William and 
Flora Hewlett Foundation, National Commission for Employment Policy, AT&T Foundation, Exxon 
Corporation, ARCO Foundation, Aetna Fourdation, The Chase Manhattan Bank, and Stuart 
Foundations. Funding from this consortium also enabled MDRC to award a modest $25,000 grant 
to each site. 

As discussed earlier, the funding structure shaped the character of the demonstration at the 
local level in two important ways. First, the JOBSTART program operated within existing agencies 
and programs under the rules and performance standards of Title IIA of JTPA or, for the 
nonresidential Job Corps Centers, under Title IVB of JTPA. It proved a serious challenge for the 
non-Job Corps sites simultaneously to follow the demonstration guidelines, the rules of Title IIA of 
JTPA, and the provisions in their contracts with SDAs. 

Second, without special funding, sites could not be expected to make major changes in their 
existing programs, limiting the extent to which the JOBSTART curriculum and instructional methods 
could be standardized. Consequently, MDRC gave sites general guidelines for program operation 
specifying the type and duration of required components of the program (education, occupational 
training, job placement, and support services). Even within this flexible framework, however, some 
program operators faced major implementation challenges. For example, some sites normally offered 
only basic skills education or vocational training; the demonstration called for both, requiring them 


3’The Defense Authorization Bill of 1992, for example, contains the Civil Community Corps 
Demonstration Program for youths, which includes summer and year-round programs in both residential and 
nonresidential settings. 

38For example, numerous proposals have been offered to enhance the apprenticeship programs in the 
United States to help more young people find stable employment. 

See Auspos, 1987, for a discussion of the origins of JOBSTART and its early implementation. 
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either to add a whole new kind of activity or to link up with other local agencies providing it. Some 


sites also had to adapt to a younger and less skilled student body than they normally served. The lack 
of special program funding also limited the extent to which non-Job Corps sites could offer the array 
of support services that were a part of the Job Corps program. 

The demonstration was thus a hybrid: part evaluation of existing programs and part test of a 
new program. The basic program differed from site to site in myriad details, but the variety did 
permit a test of how a scaled-down Job Corps-type program could operate under existing rules in 
different kinds of established agencies. If the demonstration showed positive results, it would be 
easier to replicate the program widely. 

A. The Sites 

MDRC staff recruited 13 sites (listed in Table 1.3), each of which they thought could (1) meet 
the JOBSTART program guidelines with little or no technical assistance except on techniques of 
client outreach and retention; (2) assemble sufficient operational funding for the full array of 
JOBSTART services (a significant barrier, as discussed above); and (3) yield a target of 200 sample 
members.“ All had experience running programs that included some or all of the components of 
the JOBSTART model or working with young dropouts. 

While all agreed to implement the JOBSTART model, the sites brought to the demonstration 
varying operating experiences, as shown in Table 1.3: 

¢ Sponsoring organizations. The participating organizations included adult 

vocational schools, a community college, community-based organizations that focus 
on literacy development and GED preparation, community-based organizations that 


focus on occupational skills training, and the nonresidential components of three 
Job Corps Centers.*! 


© Prior service emphasis. Some sites previously had offered only basic education 
and no skills training, while others had offered both but had emphasized skills 
training. The education-focused sites may have attracted youths who were 
primarily interested in basic education rather than skills training. Similarly, some 
sites with strong histories of skills training may have attracted youths who were 
primarily interested in learning the skills needed for a particular occupation rather 
than attaining a GED. 


“See Auspos, 1987, and Auspos et al., 1989, for a detailed discussion of the characteristics of the sites 
in the demonstration. 

“'The Job Corps Centers operated their usual nonresidential programs. Thus, they offered all 
JOBSTART services plus other Job Corps services that are not part of the JOBSTART model. 
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TABLE 1.3 


THE JOBSTART SITES 
Agency Name Location Type of Organization —_—_— Prior Service Emphasis® JOBSTART Program Structure” 
Allentown Youth Services Buffalo, NY Community-based Education Sequential/brokered 
Atlanta Job Corps Atlanta, GA Job Corps Center Education and training Concurrent 
Basic Skills Academy (BSA) New York, NY Community-based Education Sequential/brokered 
Caphol Region Education Hartford, CT Community-based Education Sequential/brokered 
Council (CREC) 
Center for Employment San Jose, CA Community-based Training with some Concurrent 
Training (CET) education 
Chicago Commons Association's Chicago, IL Community-based Training Concurrent 
Industrial and Business 
Training Programs 
Connelley Skill Learning Pittsburgh, PA Adult vocational Education and training Concurrent 
Cemter school 
East Los Angeles Skills Monterey Park, Adult vocational Education and training Concurrent 
Center CA school 
El Centro Community College Dallas, TX Community college Education and training Sequential/in-house 
Job Training Cent 
Emity Griffith Opportunity Denver, CO Adult vocational Education and training Concurrent 
School (EGOS) school 
Los Angeles Job Corps Los Angeles, CA _Job Corps Center Education and training Sequential/in-house ie 
Phoenix Job Corps Phoenix, AZ Job Corps Center Education and training Concurrent 
SER/Jobs for Progress Corpus Christi, Community-based Training Concurrent 
™ 
(continued) 


TABLE 1.3 (continued) 


NOTES: “€ducation’ refers to basic education, often as preparation for the GED examination. ‘Training’ refers to instruction in 
occupational skilis needed for specific jobs. 
‘Concurrent programs offer basic education and occupational training concurrently from the beginning of participation. Sequential/in-house 
programs offer basic education followed by occupational training, with both components provided in-house by the agency. Sequential/brokered 
provide basic education and then serve as a broker for occupational training, referring participants to other agencies. 
in October 1990 this site was renamed The Clarkson Center, inc. 
in September 1988 this site was renamed the Edmund J. Kahn Job Training Center. 


This diversity among sites led them to implement the basic JOBSTART program components 
in several ways. Eight sites were able to offer both education and training in-house and chose to 
provide them concurrently, with participants active in both activities from the start. Two sites 
provided both activities in-house, but offered them in sequence, with skills training following 
education. The remaining three sites did not have the capacity to offer skills training and chose to 
provide basic education themselves and work with other agencies to place their participants in 
subsequent occupational training elsewhere. 


B. The Program Guidelines 

Drawing on the lessons of the Job Corps and applying them within the constraints of JTPA, the 
demonstration developed a new alternative program offered in a nonresidential setting with fewer 
support services available to participants. The key elements, shown earlier in Table 1.1, included the 
core components of the Job Corps (basic education, occupational training, and job search) but a less 
extensive system of support services and no paid work experience.‘? In some respects (the 
definition of the target population and the requirement that certain activities be included), the 
program model was quite specific, while in others it allowed for considerable variation. The model 
set requirements as to the type and intensity of education and training services that were to be 
offered to participants, and it placed strong emphasis on the need for strategies to increase program 
retention. However, as mentioned earlier, sites were given a great deal of flexibility in implementing 
these core requirements. 

1. Target group. Since the program was designed to reach a population largely unserved 
by existing programs, eligibility requirements were quite specific. Participation was limited to school 
dropouts who were between 17 and 21 years of age, did not have a high school diploma or GED, 
read below the eighth-grade level, and satisfied the JTPA definition of “economically disadvantaged” 
(defined primarily by household income or reccipt of public assistance). Recognizing that program 
operators needed to meet enrollment and performance standard targets, however, the guidelines 
allowed for up to 20 percent of participants to read at or above the eighth-grade level. 

2. Education and training. The demonstration sought to test an intervention that would 
be reiatively intensive and lengthy compared to the usual JTPA activities and that would address the 
multiple deficits in participants’ skills. As a result, the program model required sites to offer a 


42Chapter 3 of this report provides more detail on the JOBSTART program model as implemented by 
the sites in the demonstration. A fuller description is given in Auspos et al., 1989. 


22. 


75 


specified minimum amount of both basic education and occupational training to provide the young 
people with a real opportunity to enhance their skills.“? The 200-hour minimum of education was 
based on an estimate of what would be needed to bring the basic skills of most participants reading 
below the eighth-grade level up to the point where they could qualify for a GED or enter 
occupational skills training. The 500 hours of training was a compromise between the very lengthy 
training that research suggested was useful and what was practical in most JTPA environments. 

Given the difficulty of keeping young people engaged in a program for an extended period and 
the competing demands on their time (including their need for income and their child care 
responsibilities), staff recognized that not all participants would complete these activities and that the 
total time in the program would be a year or less. 

The occupational skills component required classroom rather than on-the-job training, in the 
belief that participants would benefit from the intensive, closely supervised instruction possible in a 
classroom setting. Again, no specific curriculum was required. Recognizing the advantages of 
applying learning to practical problems, however, the program model required that the training 
include a combination of theory and hands-on experience. Seeking to increase the chances of 
placement following training, the program model required that the training prepare participants for 
jobs in high-demand occupations and be developed in cooperation with local representatives of the 
private sector. 

3. Support services. Attracting and keeping disadvantaged youths in education and training 
programs is a challenging problem, and the sites were expected to assist participants with 
transportation and child care. They were also encouraged to develop a package of other support 
services to facilitate program participation; the Job Corps sites offered considerably more support 
services than did the others.“ 

4. Job placement assistance. The guidelines required sites to identify possible 
training-related jobs for participants and to assist them in securing employment, but were not specific 
about how this should be done. All sites instructed the youths on work disciplines, employer 
expectations, and job search techniques, but the intensity of this effort ranged from informal guidance 
by counselors and other staff to more than 50 class hours in one site. Seven sites offered some form 
of work experience or internship (both paid and unpaid) to improve job skills. All sites provided 


“3The program model did not specify any particular curricula, though it did encourage — but not require 
- sites to offer computer-assisted instruction. 
“Job Corps Centers offered health services, recreational activities, and on-site food service, and more 
intensive counseling and peer support than did most other sites. 
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assistance in seeking employment when the youths left the program, although in two of the three 
sequential/brokered sites (CREC in Hartford and BSA in New York City), the responsibility fell 
solely on the training provider. This arrangement for job search assistance proved a serious limitation 
since, as will be discussed in Chapter 3, many young people did not reach the training phase in 
sequential/brokered sites. 

C. Key Dimensions of Program Variation 

Although the demonstration did test the JOBSTART program model, this discussion has made 
clear that there was considerable variation across the sites. JOBSTART was not the same program 
for everyone. The analysis presented in this report seeks to understand how JOBSTART "worked" 
differently for subgroups within the sample and at different types of sites. 

1. Subgroups of the sample. Much of the analysis presented in this report describes 
differences in program implementation and impacts for subgroups defined based on characteristics 
of individuals when they entered the program: for example, gender; ethnicity; age; prior education, 
work experience, and criminal record; parenting status; and welfare receipt. In this evaluation, central 
subgroup splits are males versus females (because of differences in prior employment and earnings), 
and — among females — women who were living with children of their own when they entered the 
program versus other women (because of different child care responsibilities). 

2. Types of sites. The previous report on the implementation of JOBSTART (Auspos et 
al., 1989) highlighted two dimensions of local variation as important influences on the program 
experience of the JOBSTART youths:* 

¢ Concurrent versus sequential education and training. Programs could offer 


youths basic education classes and vocational skills instruction at the same time (a 
concurrent model) or basic education before skills training (a sequential model). 


* In-house versus brokered services. Programs could offer youths education and 
training at the same agency, or the agency providing basic education could serve 
as a broker, helping participants who were completing the education phase to find 
appropriate training at other institutions (sequential/brokered sites). 


4SThe report also highlighted variations among the sites: whether they were serving JOBSTART youths 
in mainstream adult classes or in separate classes for youths; whether they offered computer-assisted 
instruction; and how they differed in scheduling (that is, the number of hours a day devoted to various 
activities) and in the length of their courses. Since these did not appear to have a major influence on program 
implementation or on participation by the youths, they were unlikely to have affected impacts and are not 
emphasized in this report. 
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Recent research has drawn policymakers’ attention to the implications of sequential versus 
concurrent programs and to the extent to which education and training in concurrent programs are 
truly integrated. The Minority Female Single Parent (MFSP) Demonstration, funded by The 
Rockefeller Foundation, tested different models of education and training in four local agencies. 
The evaluators argued that the one program among the four in the study with the most consistent 
positive impacts on employment and earnings (the Center for Employment Training [CET] in San 
Jose) probably achieved this result because of programmatic and organizational features that 
distinguished it from the remaining three.47 Specifically, the researchers highlighted that “the 
training design of the CET program — which emphasized training for all regardless of educational 
skill levels — offered remedial education within the context of job skill training, and accommodated 
trainees with diverse levels of educational skills.“ 

While the difference between sequential and concurrent programs is obvious, in practice, there 
is no clear distinction between integrated concurrent programs and other concurrent programs; they 
form a spectrum rather than falling into two neat categories, Among the JOBSTART sites, for 
example, CET/San Jose operated the most integrated program, with basic skills instruction being 
offered within the context of vocational training. Chicago Commons, another site, operated a 
partially integrated program and offered several training courses requiring technical knowledge and 
mathematics skills. Even after imposing entrance requirements among the most stringent in any 
JOBSTART site, Chicago Commons found that the skills deficiencies of some participants were too 
severe to be addressed within the integrated training context. Thus, the site also offered a separate 
basic education class. Other concurrent sites such as SER/Corpus Christi and Connelley in Pittsburgh 
operated separate education classes aimed at preparing people to pass the GED examination.‘? 


“+See Gordon and Burghardt, 1990, and Burghardt et al., 1992. 

47The CET site in the MFSP demonstration enrolled minority female single parents, whose average age 
was 28, and served them in San Jose and several other East Bay communities. The San Jose program was also 
a site in JOBSTART, but only 10 mothers became part of the JOBSTART sample at CET/San Jose, and they 
were considerably younger than the MFSP sample. 

“Gordon and Burghardt, 1990, p. xvi. The authors also cited a number of other factors that were unique 
to CET among the four sites and that they believed contributed to its large impacts, including its financial 
stability and experienced staff; integration of the MFSP program into an ongoing training operation with a 
similar mission; large scale, which allowed for training in a variety of occupations in demand in the local 
economy; attention paid to job placement; and availability of on-site child care. The remaining three sites in 
the MFSP demonstration emphasized “the acquisition of basic skills befure entry into job skill training” (p. 
xxvi) — that is, “sequential programs” using the terminology in this report. 

sites tried to coordinate the activities in education and training classes via conferences among 
the instructors and inclusion of basic skills instruction in some training classes. 
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As discussed earlier, the JOBSTART sites included agencies that provided all services in-house 


and others where the sponsoring organization served as a broker, arranging some services for 
participants at other organizations. When these two site groupings are combined, this yields three 
types of sites (sequential/in-house, sequential/rokered, and concurrent), which are analyzed 
separately in Chapter 3, on JOBSTART services. 

There is a serious complicating factor in analyzing the reasons for differences in program 
implementation and impacts across sites or groups of sites: The young people were not randomly 
assigned to different programmatic approaches. Instead, the sites offering these sequential/in-house, 
sequential/brokered, and concurrent programs operated in different kinds of settings and local labor 
markets” and served different types of youths with varying backgrounds and interests! Thus, 
the structure of the program was not the only difference among sites. Given these many differences, 
it is impossible to isolate the influence of one factor — such as concurrent versus sequential program 
structure — on program effectiveness.°? As discussed in more detail in Chapter 2, caution must 
therefore be exercised in making cross-site comparisons of program implementation and impacts. 


The evaluation of JOBSTART is divided into three main components. The first deals with the 
sites’ implementation of the program. Launching JOBSTART, the initial report on the demonstration, 
discussed site selection and characteristics, the operation of the program within JTPA, and early 
implementation experiences.*? A second report, Implementing JOBSTART, completed the imple- 
mentation analysis by describing the content of JOBSTART activities, the participation patterns of 


For example, the JOBSTART programs operated in very different labor markets. The unemployment 
rates in the sites’ metropolitan areas varied from a low of 3 percent in 1987 in Hartford, where CREC was 
located, to 12 percent in 1986 in Corpus Christi, where SER operated. Youth unemployment rates varied 
from 6 percent in 1986 in Hartford to 27 percent in 1985 in New York City, where BSA was located. 

5\Chapter 3 presents information on the characteristics of the sample in each site. 

52To rigorously compare the impacts of different programmatic approaches, more than one approach 
would have to be operated in each site, and youths would have to be randomly assigned to one of them. Even 
with this design, if the programs differed on several dimensions, it would still be impossible to isolate the effect 
of any one dimension. This type of research has rarely been undertaken. Examples include MDRC’s study 
of the impacts of job search alone versus job search plus community work experience in San Diego (Goldman 
et al., 1986), Mathematica Policy Research's study of alternative reforms of the Unemployment Insurance 
system (Corson et al., 1989), and MDRC's ongoing study of the Job Opportunities and Basic Skills Training 
(@OBS) Program, which involves comparisons of various service strategies. 

S3auspos, 1987. 
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the young people in the program, and operational lessons to be drawn from the demonstration. 


The second component of the evaluation is an analysis of program impacts. Findings based on 
two years of follow-up were reported in Assessing JOBSTART, while final impact estimates based on 
four years of follow-up form the core of this report. The research was designed to separate out the 
effects of JOBSTART itself from events attributable to other factors (such as other services 
participants were receiving and events in their lives outside the program). To accomplish this, all 
people who applied for JOBSTART and were found to be eligible were randomly assigned to either 
an experimental or a control group. Those in the experimental group were given access to the 
JOBSTART program services; those in the control group were not, although they could receive other 
services offered in their community. Since the youths were assigned at random to the two groups, 
they were similar except for the fact that only the experimental group could receive JOBSTART 
services. This type of analysis is often called “experimental” research because of its reliance on the 
methods of classical scientific experiments. Individuals in both groups were scheduled to be surveyed 
12, 24, and approximately 48 months after being randomly assigned. (The time frame for applying 
to JOBSTART varied from site to site but ranged overall from August 1985 through November 1987. 
Hence, the fielding of each wave of the survey also extended over many months.) Using these 
surveys, the experiences of the two groups can be compared to estimate the effect of the program 
on educational attainment, employment, earnings, use of public benefits, and other outcomes. 

Estimates of the impacts of the progran for subgroups of the sample defined based on pre- 
random assignment characteristics are also straightforward, since the control group counterparts of 
members of the experimental group can be identified. For example, in a subgroup analysis by gender, 
males in the experimental group are compared to males in the control group. 

The third component of the evaluation assesses the cost-effectiveness of the program. Assessing 
JOBSTART included an analysis of the costs of providing the JOBSTART services and the sources 
of variation across the sites. Chapter 7 of this report summarizes this analysis and presents an 
analysis of the net benefits and costs of JOBSTART from a variety of perspectives. 

As a guide to what is ahead, Chapter 2 of this final report on JOBSTART presents the key 
research questions and the research design used to address them. It also includes information 
showing that the sites did succeed in recruiting the young, economically and educationally 
disadvantaged youths targeted by the demonstration. Chapter 3 discusses the implementation of the 


54auspos et al., 1989, These findings are summarized and updated in Assessing JOBSTART (Cave and 
Doolittle, 1991). 
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JOBSTART program, concluding that most youths in JOBSTART participated in education and 
training activities more than the typical participant in JTPA-funded prograins, and nearly as much as 
the typical Job Corpsmember. Here, as in all subsequent chapters, the report seeks to understand 
the overall results by examining whether and how JOBSTART operated differently for key subgroups 
of youths in the sample defined by pre-random assignment characteristics. Chapter 4 examines 
educational outcomes, especially the degree of experimentals’ participation in education and training 
compared to that of controls, and whether JOBSTART led to increased attainment of a high school 
diploma or GED during the four years of follow-up. Chapter 5 examines the indications of how this 
investment in “human capital" affected youths’ employment and earnings, while Chapter 6 presents 
impacts on other outcomes such as welfare receipt, arrests, childbearing, and “productive activity," 
defined as time spent working or in education or training. An analysis of the net costs of 
JOBSTART and the benefits produced by the program is summarized in Chapter 7. In conclusion, 
Chapter 8 discusses the implications of the JOBSTART evaluation for future program design, 
operation, and research. 
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CHAPTER 2 
THE JOBSTART EVALUATION AND SAMPLE 


This chapter describes the research design and the sample of young school dropouts who were 
involved in the study. Section I indicates how the study was designed to answer the research 
questions posed in Chapter 1, with a special emphasis on the random assignment approach for 
assessing the difference the program made in the lives of the young people who participated in 
JOBSTART. Section II describes the characteristics of the research sample overall and of different 
subgroups of the sample, defined by their site, gender, and other characteristics of interest to 
policymakers and program planners. 


I. An Overview of the Study Design 

Although education and training services for young school dropouts are limited, some youths 
who entered JOBSTART would have received a high school diploma or GED certificate, found a job, 
increased their earnings, or left the welfare rolls on their own even if they had not been in the 
program. As noted in Chapter 1, to isolate the impact of JOBSTART from other factors that might 
have produced such outcomes, MDRC randomly assigned applicants to experimental and control 
groups. The two groups were similar except that only the experimental group could receive 
JOBSTART services. Comparison of the two groups’ experiences during the four years after random 
assignment (the follow-up information available for this report) thus provides a reliable estimate of 
the difference that the program made.! 


A. How Random Assignment Was Conducted 
Figure 2.1 shows the steps in the client intake and random assignment process.? Youths who 
expressed an interest in program services entered the program through a process that took from one 


1 Appendix A details the sources of data used in comparing the experiences of the two groups: enrollment 
forms completed just prior to random assignment; a management information system (MIS) that provided data 
on participation in the program; results from the Test of Adult Basic Education (TABE) administered to 
members of the experimental group; follow-up surveys for this evaluation conducted 12, 24, and 48 months 
after random assignment; data on program costs from a variety of sources; and qualitative data based on 
interviews with program staff, field observations of program operations, and focus group discussions with 


participants. 
2For more detail sec Auspos et al., 1989. 
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FIGURE 2.1 


THE JOBSTART EVALUATION DESIGN 
AND SAMPLE FLOW 
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Informational Interview 
——— Drop-of! 


JTPA/Job Corps Certification i 
Enrollment Forms 
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Concurrent Sites Sequential Sites 


Basic Education No JOBSTART Services 


and but Eligib!e for 
Other Services 


Occupational Skills 
Training 


SOURCE: Adapted from Auspos et al., 1989. 
NOTE: Support services such as child care and transportation were offered at both concurrent and sequential sites. 


-30- 


§3 


day to one month (10 days on average), depending on the site.> Most of the steps were part of the 
usual JTPA Title IIA (or, in Atlanta, Los Angeles, and Phoenix, the Job Corps) intake procedures; 
in most sites only the reading test and random assignment were added for the JOBSTART 


Demonstration. The order of the steps varied from site to site, as did the division of responsibility 
between the JOBSTART program operator and the local JTPA service delivery area (SDA). The 
process included: 


* Client recruitment. JOBSTART was voluntary, so the JOBSTART program 
operator and, in some cases, the local SDA actively recruited youths to apply, using 
a variety of techniques to meet their enrollment goals. Program staff approached 
potential recruits through media announcements; mailings to dropouts and welfare 
recipients; and outreach visits to schools, parks, and other youth gathering places. 
They distributed posters and fliers advertising program services and sought referrals 
of eligible youths from JTPA, community organizations, schools, and social service 
agencies. Recruitment activities frequently took staff members beyond the walls 
of their office and their rine-to-five workday. Recruitmer.t through public school 
referrals or outreach was productive in the school-based JOBSTART programs. 


* Informational interview. In a brief interview, JOBSTART staff would explain to 
a potential applicant the program's services and obligations and, often, the random 
assignment procedures. Some sites also regularly included a tour of their facilities 
to help recruits understand program services, opportunities, and demands. 


¢ Assessment. Program staff assessed whether applicants met the age (17 to 21), 
educational status (school dropout), and income requirements for JOBSTART. 
They also ascertained the youths’ needs for support services and their 
appropriateness for the program, screening out those with problems the program 
was not equipped to handle. The assessment process was relatively extensive at the 
si‘es operated by Job Corps, which had the broadest array of support services. Job 
Corps staff assessed recruits for emotional problems, drug and alcohol abuse, 
trouble with the law, unstable living situations, health problems, and motivation. 
Other sites screened mostly to identify youths who were likely to prove dangerous 
or disruptive, such as those with evident drug or alcohol problems. 


© Reading test. Most program operators tested recruits early in the intake process 
to determine whether their reading skills were below the eighth-grade level, as 
required by JOBSTART eligibility criteria. Four sites (the three Job Corps 
Centers and CET/San Jose) delayed testing until later in the program, limiting their 
testing to participants. As noted earlier, all the sites were permitted to enroll up 
to 20 percent of their recruits with higher reading scores to help meet enrollment 
goals, Some sites also set a lower limit of a fourth-, fifth-, or sixth-grade reading 


2Sites varied greatly in the amount of initial assessment they conducted before allowing entry into the 
program. There was also wide variation in state and local interpretation of the documentation needed to 
establish eligibility for JTPA Title IIA programs. 
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level. Staff at those sites believed that the youths would need to read at least at 
those levels to benefit from the education and training services that were available 
locally. 


© JTPA Title ILA/Job Corps certification. Recruits had to prove that they met the 
eligibility criteria for JTPA Title ILA-funded services. At the three Job Corps sites 
recruits also had to meet Job Corps eligibility criteria. At all the sites certification 
of eligibility required proof of residency, age, and economic disadvantagedness. 
SDAs at most sites required applicants to provide supporting documentation of all 
aspects of JTPA Title IIA eligibility for approval of enrollment into JOBSTART. 
Local regulations and practices concerning the certification process strongly 
influenced the speed and ease of certification. In fact, program operators at six 
sites pinpointed JTPA Title ILA certification procedures as a major bottleneck in 
the intake and enrollment process. 


¢ Informed consent form, enrollment form, and random assignment. After a staff 
member described the random assignment process, the applicant would sign an 
informed consent form, agreeing to accept the results of random assignment and 
to cooperate in follow-up survey interviews. At that point, program or SDA staff 
filled out the enrollment form, using information provided by the applicant, and 
then telephoned MDRC, where the random assignment was made. Youths 
assigned to the experimental group were told to report to classes or, in some sites, 
to an orientation session. Program staff would contact experimental group 
members (“experimentals") who did not appear for program activities, encouraging 
them to participate and assisting them with needed support services. Applicants 
assigned to the control group would be reminded that they were part of the 
research project and would be contacted later for the follow-up surveys. They 
were also told that they could seek services elsewhere on their own. 


A total of 2,312 people were randomly assigned: 1,163 to the experimental group and 1,149 to 
the control group. The sites conducted random assignment over varying time periods. Connelley 
Skill Learning Center in Pittsburgh enrolled the first sample members in August 1985, and the Los 
Angeles Job Corps enrolled the last sample members in November 1987. Open-entry/open-exit sites 
continuously recruited applicants to maintain enrollment levels, while sites operating fixed-cycle 
programs intensified recruitment efforts before the start of a cycle of classes. Overall, sites reported 


‘In many sites, program staff would often provide sample members with a list of other services in the 
community at the time of random assignment. Individuals who ended up in the control group may have used 
that list to seek alternative services. As discussed later in this chapter and elsewhere in the report, however, 
the experimental group had much higher levels of participation in services than did the control group during 
the first two years of follow-up. 

SAll but one of the 2,312 youths who were randomly assigned had completed enrollment forms, which 
provided the pre-program baseline data on age, gender, previous employment, extent of schooling, and other 
characteristics that were used in the evaluation. Such data were used to define key subgroups within the full 
sample and to adjust the impact estimates for any experimental-control group differences in baseline 
characteristics. In the remainder of this report, we therefore cite 2,311 as our “full research sample.” 
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that about 89 percent of the 1,163 youths in the experimental group participated to some extent in 
JOBSTART services. The percentage participating did vary among the sites, from a high of 100 
percent at Allentown in Buffalo and El Centro in Dallas to a low of 64 percent at CET/San Jose. 
(Chapter 3 presents more detailed information on participation by site.) Four factors influenced the 


percentage of experimentals reported to be active in the program: 


¢ Length of the intake process. The process of recruitment into the JOBSTART 


Demonstration took a relatively short time in many sites, often less than a week. 
At a few sites, however, the extended checks of eligibility (particularly at the Job 
Corps sites) meant that intake lasted much longer, and in the meantime some 
youths who were eventually assigned to the experimental group found other 
program opportunities or lost interest. 


Open-entry/open-exit versus fixed-cycle scheduling. Open-entry programs allow 
young people to enter and finish at any time, while other programs operate on 
fixed schedules of class cycles.° Youths assigned to the experimental group in 
fixed-cycle sites might face delays in program start-up, resulting in lower 
participation rates. 


Start-up or scheduling problems. Some sites had unexpected problems getting 
youths into services. The most notable example was the experience of the early 
entrants at CET/San Jose, where program slots were not available for up to a 
month after random assignment because of funding cuts. This delay contributed 
to this site’s experimentals having the lowest rate of participation in JOBSTART 
services, although, as shown in Chapter 4, the experimental-control difference in 
service receipt was as great at CET as it was for the entire sample. 


Differences in the sites’ attendance reporting. The program elements counted in 
participation at all the sites were education, training, and other activities such as 
life skills training, work experience, and — in the Job Corps sites — a lengthy 
orientation. Participation in an extended assessment of training interests was not 
included in hours at CET/Sai Jose, however. As a result, if youths at CET 
attended this assessment and nothing else, their reported number of hours was zero 
and they were counted as nonparticipants. This difference in reporting practice 
may have affected CET/Sau Jose's participation rate and reported hours in 


B. The Research Sample Used in This Report 


Follow-up surveys conducted at 12, 24, and 48 months after random assignment gathered data 
on outcomes such as participation in education and training programs, educational attainment, 


©The sites operating open-entry/open-exit programs were Allentown in Buffalo, the Atlanta Job Corps, 
CET/San Jose, the East Los Angeles Skills Center, El Centro in Dallas, the Los Angeles Job Corps, and the 
Phoenix Job Corps. EGOS in Denver offered classes on a semester schedule but allowed entry whenever 
Classes were in session. The remaining five sites — BSA in New York, CREC in Hartford, Chicago Commons, 


Connelley in Pittsburgh, and SER/Corpus Christi - operated fixed-cycle programs. 
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employment, earnings, and receipt of public assistance.’ Of the 2,311 youths in the full research 
sample, 1,941, or 84 percent, responded to the 48-month follow-up survey and constitute the "48- 
month impact sample" analyzed in this report.® The experience of the 988 experimentals in this 
sample also serves as the basis for the implementation analysis in Chapter 3, which examines such 
issues as participation rates in JOBSTART and its components, and hours and duration of 
participation.” 


For this study to produce unbiased estimates of program impacts, several conditions had to be 
met. These are addressed in the following questions. ! 

1. Did random assignment result in a group of experimentals with the same measured 
pre-program characteristics as the control group? Random assignment — if properly implemented 
— should create a group of JOBSTART experimentals with the same characteristics at the start of 
the program as the controls, on average, so that any observed differences between the experimentals 
and the controls in post-random assignment behavior will provide unbiased estimates of program 
impacts.'! The information presented in Appendix B (Tables B.1 and B.2) for the 1,941 people 
randomly assigned for whom there were 48 months of follow-up data shows that there were virtually 
no measured differences in characteristics between the experimentals and controls. !2 


Tas noted in Appendix A, information on participation in JOBSTART was provided by the sites as part 
of a special management information system (MIS) created for the demonstration. The follow-up surveys 
collected information on participation in all other education, training, and employment programs for both 
experimentals and controls. 

®This impact sample is larger than that responding to the 24-month follow-up survey (1,839, or 80 percent 
of the full sample). Some individuals not responding at 24 months did respond at 48 months, at which time 
they responded to questions on the earlier period. 

*The 48-month impact sample for this report is slightly different from the sample in our previous reports 
on JOBSTART because some who responded to the earlier surveys did not respond to the final 48-month 
survey, and some who responded to the final survey did not respond to an earlier survey. Thus, the findings 
in this report on program participation and impacts during the first 24 months of follow-up are slightly 
different from those presented in our earlier reports. Nevertheless, the shifts in sample have not materially 
affected the basic story. 

10For a fuller discussion of some of these issues, see Appendix B. 

1This condition is known as the “internal validity” of the estimates. 

12The only difference that was statistically significant was that experimentals in the 48-month impact 
sample were slightly more likely than controls to be a part of an AFDC case headed by another member of 
their household (see Table B.2). 

On a site-by-site basis, the 48-month impact sample consists of 26 separate groups of experimentals and 
controls. If experimentals in any site are compared with controls in that site, the internal validity of site 
impacts may be assessed. As would be expected in 13 relatively small subsamples of the full 48-month impact 

(conti 1ued...) 
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2. Are those 1,941 sample members with 48 months of survey data representative of the 
entire JOBSTART sample of 2,311? Forty-eight months of survey data are available for nearly 84 


percent of all the youths who were randomly assigned, including 85 percent of experimentals and 83 
percent of controls, 88 percent of women who were custodial mothers at baseline, 88 percent of other 
women, and 80 percent of males. Appendix Table B.3 shows that there are some statistically 
significant differences between those who responded to the surveys and nonresponders, but in 
characteristics other than experimental status. In the full sample, responders were less likely than 
nonresponders to be male. Responders were more likely than nonresponders to have entered the 
sample at the Allentown (Buffalo), CREC (Hartford), or El Centro (Dallas) sites, to be white or 
Hispanic, to be older, to have left school during grade 11 or 12, and to have lived with both parents 
at age 14. 

When nonresponse is randomly distributed among members of both the experimental and 
control groups, it is troublesome only because it reduces the sample size and thus the statistical power 
to find impacts of a given magnitude.!> However, when nonresponse is greater among one research 
group (which is not the case here) or among members of either research group who have certain 
characteristics (such as men), impacts may be biased slightly unless they are corrected for 
nonresponse. The impacts presented in this report do not include any corrections for the differences 


between survey responders and nonresponders. !4 


The success of attempts to implement such 
corrections is uncertain, and the differential response rates found do not seem large enough to 
warrant such measures, which could introduce biases of their own. The high overall response rate 
of 84 percent makes findings on the 48-month impact sample representative of a very broad group 
of the full sample. 

3. Did most experimentals receive JOBSTART services, and did relatively few controls 
receive them or any equivalent services? For these conditions to be met, experimentals must 
participate in JOBSTART and controls must be excluded from JOBSTART and not find equivalent 
services elsewhere in their community. As discussed above, nearly 90 percent of experimentals were 


active in JOBSTART. Because of successful implementation of random assignment procedures, 


12(, continued) 
sample of 1,941, there are a few experimental-control differences in demographic characteristics within 
individual sites. 

Randomly distributed nonresponse does not alter the expected values of adjusted mean outcomes, and 
thus does not bias impacts. 

4 most flexible correction for nonresponse is incorporation of an additional equation for survey 
response into a two-equation system with the impact equation. 


-35- 


&8 


virtually no controls were served in JOBSTART programs. In addition, analysis of the receipt of 
education and training services from JOBSTART and non-JOBSTART sources indicates that controls 
did not find an equivalent level of services elsewhere (see Chapter 4). For example, in the first year 
after random assignment, 90 percent of experimentals but only 26 percent of controls participated 
in some type of education or training activity. As this and the other measures used in Chapter 4 
indicate, experimentals did receive a noticeably greater total amount of employment and training 
services. But it is important to keep in mind that controls were not an unserved group; many 
received substantial services from sources other than the JOBSTART programs. Therefore, the 
impact findings presented in Chapters 4 through 6 of this report should be interpreted as measuring 
the incremental impact of the services received by experimentals above those received by controls. 

4. Do the impacts per person assigned to the experimental group differ greatly from the 
impacts per person participating in JOBSTART? Some of those who were randomly assigned to 
the experimental group (the group given access to the JOBSTART program) never participated. 
However, they still were included as part of the experimental group when average impacts were 
calculated, somewhat “diluting” the impacts.!5 Fortunately, the percentage of nonparticipants was 
small (only 11 percent of the 988 experimentals in the 48-month impact sample), so including them 
diluted the impacts only slightly."© In other words, while the impacts refer to all surveyed 
experimentals (nonparticipants as well as participants), they would be only slightly changed if adjusted 
to include surveyed participants only. a 


1SIf the nonparticipants had not been counted, the experimental group would no longer have been truly 
comparable to the control group. Including them in the impact calculations was designed to avoid a form of 
“selection bias" — in this case, caused by those who had “selected themselves" out of their chance to join the 
JOBSTART program or were discouraged by program staff. 

I6See B for details on such adjustments. In some sites, nonparticipation rates were considerably 
higher than the 11 percent for the entire 48-month impact sample, so the difference between impacts per 
experimental and per participant is greater. 

17in addition to the issues discussed above, the study posed some important methodological issues 
concerning data collection. Most important among them was the problem of abrupt changes in some key 
variables at the “seam” between survey waves. For example, employment and earnings reported for month 12 
in the 12-month survey were noticeably higher than employment and earnings for month 13 in the 24-month 
survey. A similar shift occurred at the month-25 seam between the 24- and 48-month surveys. This problem 
is likely to have arisen owing to recall problems in the later surveys (at 24 and 48 months), since the 
respondents were providing information on a period one or two years in the past. Sample members who were 
*missed” in the 24-month survey and provided information on the entire 48-month period at one time did not 
exhibit an abrupt change in employment and earnings levels. Appendix A discusses how this problem was 
handled in the JOBSTART evaluation. Similar problems have been encountered in other studies; see 
Burghardt et al., 1992, Appendix D, for another example. 
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Examining the pre-program experiences and characteristics of the young people in the 
JOBSTART sample is important for three reasons. First, it shows whether the sites succeeded in 
enrolling economically disadvantaged young people with poor skills who were the target group for 
the demonstration. Second, it permits a comparison of the JOBSTART youths with those served by 
other federally funded employment and training programs. Third, much of the analysis in this report 
moves beyond results for the full sample of JOBSTART youths to examine whether and how the 
program worked differently for subgroups of the sample (especially males, custodial mothers, and all 
other women), and understanding the pre-program characteristics of these subgroups is the first step 
in such an analysis. This third reason is important because subgroups defined by a single 
characteristic (such as gender, age, prior employment, or the type of program model at the site to 
which they applied) may vary in other characteristics as well. Females in the sample, for example, 
may have had less prior employment experience and more prior public assistance receipt than the 
men in the sample. Sites offering education followed by training at another agency may attract 
different applicants than those known for their training courses. Understanding the combination of 
characteristics associated with subgroups helps prevent misinterpretation of any observed differences 
in program participation and effectiveness. With this goal of the report in mind, the remainder of 
this chapter summarizes the characteristics of the youths at each site and of key subgroups that will 
be examined later in the report. 


A. Characteristics of the 48-Month Impact Sample 

Table 2.1 provides detailed background information on the 48-month impact sample of 
JOBSTART youths and indicates that the sites in the demonstration succeeded in recruiting the 
intended target group. The column labe!ed “all 13 sites” shows the characteristics of the entire impact 
sample; the remaining columns are discussed in Section IIB of this chapter. The sample is made up 
of slightly more women than men; most of the sample are members of minority groups and were 
unmarried at baseline; nearly 75 percent were under 20 years of age at baseline; slightly less than half 
had not worked during the year prior to random assignment; and about 60 percent left school before 
the 11th grade.!8 


18The only real divergence of the sample from the intended target group occurred because a slightly 
higher than planned percentage of the youths read at the eighth-grade level or above. This happened because 
some sites tested reading skills after random assignment and only for experimentals. For this reason, sample 
members’ reading levels are not included among the baseline characteristics in this report. 
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Yes 

Men who have own child(ren) 
No 
Yes 


Sample size 


All 
13 Shes 
(%) 


53.6¢** 
46.4 


Boers 

443 

43.6 
3.2 


§.0*** 
23.2 


Chicago = Conneliley 
Job Corps San Jose Commons (Pittsburgh) 


Atlama CET/ 
(%) (%) 
59.4 49.7 
40.6 50.3 
29 15.0 
97.1 6.0 
0.0 70.1 
0.0 9.0 
29 9.0 
56.5 24 
0.0 37.1 
0.0 1.2 
0.0 60 
40.6 3.6 
0.0 32.9 
0.0 7.8 
27.5 44.3 
31.9 5.4 
6.2 45.5 
43 48 


TABLE 2.1 
CHARACTERISTICS AT THE TIME OF RANDOM ASSIGNMENT, BY SITE 


(%) 


413 
58.7 


67 
74.7 
18.7 

0.0 


10.7 


(%) 


East 
LA Skills 
Center 
(%) 


42.5 
57.5 


EGOS 


(Denver) 
(%) 


66.7 
33.3 


Phoenix 
Job Corps 
(%) 


§5.2 
448 


23.9 
17.2 
53.7 

5.2 


SER/ 
Corpus 
Christi 
(%) 


Parental status 


Women living with own 

child(ren) 
No 27.5*** 21.8 30.7 25.9 29.1 36.4 
Yes 26.2 31.3 28.1 32.6 18.8 28.3 

Men who have own child(ren) 
No 40.4 42.5 39.8 29.6 §1.3 29.3 
Yes 5.9 45 1.3 11.9 09 6.1 

Sample size 1,941 179 231 135 117 99 
(continued) 
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All Atlanta CET/ 
Characteristic 
and Subgroup (%) (%) (%) 
Employed within past year 
No 47.1*¢¢ 37.7 39.5 
Yes 52.9 62.3 60.5 
Prior employment, by gender 
Women employed within 
past year 
No 30.0°** 30.4 23.4 
Yes 23.6 29.0 26.3 
Men employed within past year 
No 17.1 72 16.2 
Yes 29.3 33.3 34.1 
i. Left school in grade 11 or 12 
° No §8.7¢¢* §9.4 39.5 
: Yes 41.3 40.6 60.5 
Recelved occupational training 
83.2°** 69.6 91.6 
16.8 30.4 8.4 
73.4¢** 76.8 77.8 
26.6 23.2 22.2 
9.5°** 5.8 108 
90.5 94.2 89.2 
81.1°** 88.4 86.8 
18.9 11.6 13.2 
1,941 69 167 


(%) 


56.0 
440 


26.7 
14.7 


29.3 
29.3 


48.0 
52.0 


88.0 
12.0 


42.7 
57.3 


27 
97.3 


78.7 
21.4 


75 


13 Shes Job Corps SanJose Commons (Pittsburgh) 


(%) 


26.8 
71.2 


18.5 
35.9 


10.3 
35.3 


65.2 
34.8 


66.3 
33.7 


53.8 
46.2 


43 
95.7 


74.5 
25.5 


LA Skills EGOS Phoenix Corpus 
Cemer (Denver) Job Corps Christi 
(%) (%) (%) (%) 
50.9 39.9 $45 34.4 
49.1 60.1 45.5 65.6 
25.5 31.8 38.1 243 
17.0 34.8 17.2 18.2 
25.5 8.1 16.4 10.1 
32.1 25.3 28.4 47.4 
67.9 56.1 61.9 74.41 
32.1 43.9 38.1 25.9 
84.9 93.4 94.0 69.2 
15.1 66 6.0 30.8 
78.3 76.3 85.8 70.4 
21.7 23.7 14.2 29.6 
3.8 86 11.2 28.7 
96.2 91.4 68.8 71.3 
87.7 74.2 85.8 75.7 
12.3 25.8 14.2 24.3 
106 198 134 247 


TABLE 2.1 (continued) 


Sequential/in-House 
All El Centro LA Job 


13 Stes (Dalias) Corps 
(%) (%) (%) 


Employed within past year 
No 47.1°** 45.8 749 
Yes 52.9 54.2 25.1 


Prior employment, by gender 
Women employed within 
past year 
No 
Yes 
Men employed within past year 
No 
Yes 


Left school in grade 11 or 12 
No 
Yes 


Recuived occupational training 


Sequential/Brokered 


“Allentown BSA CREG 


(Butfalo) (NYC) (Hartford) 
(%) (%) (%) 


57.0 63.2 30.3 
43.0 36.8 69.7 


6.1 
93.9 


withiri past year 
83.2*** 
16.8 
73.4*** 
26.6 
Marital status 
Ever married 9.5*** 
Never married 90.5 
Living in own household or 
with boy/girtfriend 
No 81.1*** 89.9 87.4 
Yes 18.9 10.1 126 
el 
Sample size 1,941 179 231 


re LL 


Jf 


60.0 94.0 77.8 

40.0 6.0 22.2 

135 117 99 
(continued) 
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TABLE 2.1 (continued) 


ee ee 
Concurrent 


East SER/ 
All Atlanta CET/ Chicago Connetley LA Skills EGOS Phoenix Corpus 
Characteristic 13 Sites Job Corps) San Jose Commons (Pittsburgh) Center (Denver) Job Corps Christi 
and Subgroup (%) (%) (%) (%) (%) (%) (%) (%) (%) 
Own AFOC case or receiving 
General Assistance 
No 73.1¢** 
Yes 26.9 
Own AFOC case 
No 78.4*** 
Yes 21.6 
Receiving Food Stamps 
No 62.5*** 68.1 89.2 34.7 27.7 65.1 61.1 739 68.0 
Yes 37.5 31.9 10.8 65.3 72.3 49 38.9 26.1 32.0 
Arrested since age 16 
4. No 85.0°** 85.5 76.0 813 87.5 85.8 82.8 87.3 £0.2 
ts Yes 15.0 145 24.0 18.7 12.5 14.2 17.2 12.7 198 
Arrested since age 16, 
by gender 
Women 
No 50.8*** 55.1 43.7 38.6 §1.1 40.6 61.1 54.5 405 
Yes 28 44 6.0 27 3.3 1.9 5.6 07 2.0 
Men 
No 34.2 30.4 32.3 42.7 3.4 45.2 21.7 32.8 39.7 
Yes 12.2 10.1 18.0 16.0 9.2 12.3 11.6 11.9 17.8 
Lived with both parents at 
age 14 
No 65.1*** 73.9 55.1 73.3 78.3 §5.7 596 49.3 45.7 
Yes 34.9 26.1 449 26.7 21,7 443 40.4 50.7 54.3 
) g ) Reason for leaving regular 
high schoo! 
School-related 47.7¢** 39.1 §3.9 46.7 50.0 519 42.9 54.5 45.7 
Job-related 10.1 145 14.4 20.0 3.3 17.9 9.1 5.2 13.4 l {) 0 
Other 42.2 46.4 31.7 33.3 46.7 30.2 48.0 40.3 409 
Sample size 1,941 69 167 75 184 106 198 134 247 


ee ec 
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TABLE 2.1 (continued) 


Sequential/in-House Sequential/Brokered 
All E! Centro LA Job Allentown BSA CREC 
Characteristic 13 Sites (Dallas) Corps (Buffalo) (NYC) (Hartford) 


and Subgroup (%) (%) (%) (%) (%) (%) 


Own AFDC case or receiving 
General Assistance 
No 73.1¢** 


Yes 26.9 


Own AFDC case 
No 78.4*** 
Yes 21.6 


Receiving Food Stamps 
No 62.5*** 
Yes 37.5 


Arrested since age 16 
No 


85.0*** 
Yes 15.0 


Arrested since age 16, 


Reason for leaving regular 
high school 
School-related 47.7*** 48.0 38.1 45.2 59.8 50.5 
Job-related 10.1 8.4 13.9 5.2 3.4 7.1 
Other 42.2 43.6 48.1 49.6 36.8 42.4 
Pe RS Ore Ee eT 
Sample size 1,941 179 231 135 117 99 
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TABLE 2.1 (continued) 


SOURCE: MDRC calculations from JOBSTART enroliment form and survey data. 


NOTES: Calculations for this table used data for all 1,941 sample members for whom there were 48 months of follow-up survey data. Sample sizes 
reported may fall short of this number because of tems missing from some sample members’ questionnaires. 

Distributions may not total 100.0 percent because of 

A Pearson chi-square statistic was used to test the hypothesis of equal distributions. Statistical significance levels are indicated as *** = 1 percent; 


** = 5 percent; * = 10 percent. 


The youths in the 48-month impact sample appear to have been more disadvantaged than the 
majority of youths served nationwide by JTPA Title IIA programs during the period JOBSTART was 
in operation. In the effort to serve those youths at risk of chronic unemployment, JOBSTART 
worked exclusively with school dropouts, a segment of the youth population that made up a relatively 
small proportion of JTPA Title IIA enrollees. For example, in the National JTPA Study, about 40 
percent of the males and SO percent of the females in the out-of-school youth sample had a high 
school diploma or GED. Even when the comparison of youths is limited to young school dropouts, 
it appears that JOBSTART reached a more disadvantaged population than did most JTPA Title 
IIA-funded programs.!? 

Nationwide, 80 percent of Job Corpsmembers were school dropouts in program year 1986, when 
the JOBSTART Demonstration was in operation, but their other chcracteristics suggest greater 
barriers to employment than the JOBSTART youths faced.2? Job Corpsmembers tended to be 
younger than JOBSTART sample members: 42 percent were age 16 or under in 1986, compared to 
29 percent in JOBSTART. Sixty-one percent read at the sixth-grade level or below at entry into the 
Job Corps, compared to 52 percent in JOBSTART.2! On the other hand, a higher proportion of 
JOBSTART sample members were receiving public assistance and were members of minority groups 
than were Job Corpsmembers. The residential character of the Job Corps program also introduces 
another difference: All residential Corpsmembers are willing and able to live away from home, but 
an unknown — though probably large — portion of JOBSTART sample members would not be.” 


19approximately 56 percent of the JOBSTART 48-month impact sample were receiving some form of 
public assistance at the time they entered the program (not shown in Table 2.1), compared to 39 percent of 
young dropouts served by JTPA Title ILA programs at the time and less than 35 percent of the youths in the 
National JTPA Study. Moreover, the proportion of the JOBSTART sample receiving AFDC at baseline (3S 
percent) was higher than that of young dropouts in other JTPA Title ILA programs (21 percent). This higher 
rate of welfare receipt partly reflects the fact that a greater proportion of the JOBSTART sample was female 
(53 percent), compared to the dropout group participating in other JTPA Title ILA programs (45 percent 
female). Also, minorities were much more heavily represented in JOBSTART than in JTPA Title ILA-funded 
services for young dropouts nationally. Hispanic dropouts constituted 44 percent of the JOBSTART sample 
but only 14 percent of JTPA Title ILA enrollees at the time, and JOBSTART served proportionally more black 
dropouts (46 percent) than did other JTPA Title ILA programs (34 percent). (See U.S. Department of Labor, 
Division of Performance Management and Evaluation, 1988.) 
2LJ.S. Department of Labor, Employment and Training Administration, 1987. 
2lIn JOBSTART, JTPA performance standards and practices led some JTPA Title ILA-funded sites to 
exclude youths with very low reading scores. The Job Corps sites in JOBSTART appeared to include a higher 
proportion of youths with very low reading scores than did other sites. 
¢ difficulties encountered in implementing an unsuccessful random assignment study of the residential 
versus nonresidential Job Corps programs illustrate the importance of this difference in characteristics. The 
study originally assumed that a substantial proportion of the Job Corps applicant poo! would be indifferent 
(continued...) 
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These comparisons suggest that JOBSTART sites did succeed in attracting disadvantaged young 
school dropouts, as intended in the demonstration. However, these young people were not among 
the most disadvantaged youths nationwide: only 5 percent of females and 26 percent of males 
reported at program intake that they had a criminal arrest record; most were not teenage parents; 
and about half had worked during the year before random assignment.2> Forty-eight percent 
reported they had left school for school-related reasons such as poor grades, lack of interest, or 
discipline problems. In summary, the JOBSTART youths probably fell between the typical JTPA and 
Job Corps participant in baseline skill levels and job-readiness. 


When individuals with certain characteristics are concentrated in one or a few sites, the 
influence of their individual characteristics on program implementation and impacts are "confounded" 
with the influence of site characteristics. This problem is a virtual nonissue with regard to the 
proportion of experimentals and controls at the 13 sites: All sites had approximately equal 
proportions of the two groups. Among other characteristics, however, it is an extreme issue with 
regard to ethnicity: The proportion cf black sample members at a site ranged from zero at the East 
Los Angeles Skills Center to over 97 percent at the Atlanta Job Corps, as shown in the indiv.dual 
site columns of Table 2.1. Thus, the influence of research status on implementation and impacts is 
almost independent of site, but the influence of ethnicity is much more confounded with the influence 
of site characteristics. Therefore, while comparisons of experimental-contro! differences in 
post-program outcomes for the full sample may be confidently interpreted as resulting from 
experimentals’ access to JOBSTART, there is not the same confidence about comparisons of 
participation or impacts for ethnic subgroups. These results could be heavily influenced by such site 
characteristics as program structure, labor market conditions, or other important factors completely 


22(..continued) 
as to whether they got into a residential or nonresidential program, and the study proposed to randomly assign 
members of this “indifferent” group to the two program types. This group turned out to be too small for the 
study to proceed. 

231t is very likely that youths underreported past arrests, since they were asked about this at program 
intake and may have assumed that a positive response would lower their chances of getting into the program, 
or they may not have wanted to provide this information to staff, whom they did not yet know. In addition, 
ony 106 men (12 percent of the male sample) reported being a father. 

“Overall, most sample members were black (44 percent) or Hispanic (44 percent). In five sites, over 
two-thirds of the youths were black, while in three others, over two-thirds were Hispanic. The low overall 
proportion of white, non-Hispanic sample members (9 percent) was concentrated at the Phoenix Job Corps 
and, to a lesser extent, at Allentown in Buffalo, CET/San Jose, and EGOS in Denver. 
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external to JOBSTART. Most characteristics fall somewhere between "independent" and 
“confounded” in relation to site characteristics, and unless special techniques are used to remove 
associations between site and other characteristics, impact comparisons among many subgroups and 
site groupings may be misleading. 

Fortunately, in view of the heavy emphasis in this report on comparing outcomes by gender, 
there is much less cause for concern in making comparisons of impacts for gender-defined subgroups 
than for those based on ethnicity. The validity of these comparisons is based on the fact that the 
impact sample was 46 percent male and 54 percent female overall, and men and women were 
distributed across sites much more evenly than were blacks or Hispanics.~ A more refined analysis 
of gender, appearing throughout this report, splits the sample further, based on parenting status at 
random assignment. This breakdown creates categories that can signal differences in barriers to 
employment or willingness to sacrifice in order to obtain a steady source of earned income. Overall, 
half of the women lived with children of their own; among men, about one-eighth reported that they 
already were parents at baseline. The proportion of parents among women and men in the sample 
did vary among the sites, but the variation was much less than was the case for ethnicity.” 

Site differences were large for several other subgroups of the research sample: 

¢ Sample members’ amount of prior schooling varied among the sites more than 

gender or parenting status. Large differences in baseline educational attainment 
are important to bear in mind when examining post-program GED attainment 


rates. Other factors aside, those who were closer to finishing high school at 
baseline were more likely to have received a GED during the follow-up period. 


¢ Employment during the year before random assignment varied among the sites 
even more than prior schooling. Holding all other observed factors constant, not 
having worked recently may signal cither greater barriers to employment or more 
interest in schooling than in employment.2” 


25The proportion of males in a site ranged from a high of 59 percent at Chicago Commons to a low of 
33 percent at EGOS in Denver. In addition to Chicago Commons, four other sites had mate majorities: 
SER/Corpus Christi, the East Los Angeles Skills Center, BSA in New York City, and CET/San Jose. 
proportion of men reporting fatherhood ranged from almost none at the Los Angeles Job Corps, 
BSA in New York City, and the East Los Angeles Skills Center to 19 percent at Chicago Commons. At 
Connelley in Pittsburgh, El Centro in Dallas, SER/Corpus Christi, EGOS in Denver, the Phoenix Job Corps, 
and Allentown in Buffalo, JOBSTART women were more likely to be custodial mothers, while at East Los 
Angeles Skilis Center, BSA in New York, CET/San Jose, and CREC in Hartford, women who either had no 
children or were not living with them were in the majority among females. 
27Qverall, more men than women worked in the year prior to random assignment; for each gender 
subgroup, the proportion who had worked also varied among the sites. In eight sites, the majority of women 
had not worked in the year before random assignment, with the ratio of nonworking women to working women 
(continued...) 
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¢ Public assistance receipt varied greatly. The percentage of a site’s sample 
receiving public assistance may be a good indication of the relative levels of income 
and job-readiness of the young people there. 

In subsequent chapters of this report, program impacts for subgroups of the 48-month impact 
sample are presented. These subgroups are defined based on pre-random assignment (that is, 
pre-program) characteristics, and two types of analyses are used.2? One type splits the entire 
sample into subgroups defined by a characteristic such as gender. This "split sample" subgroup 
analysis does not control in any way for other measured differences among the subgroups, such as 
in site or prior work experience. If a subgroup is concentrated in a few sites, as is the case for 
Hispanics, then the split-sample results may reflect site differences as much as subgroup differences. 
Since neither men nor women were concentrated in particular sites, the split-sample analysis 
presented for these subgroups in later chapters is appropriate. The second type of subgroup analysis 
— intended to address the problem of "confounded" influences on impacts discussed above — presents 
results that are statistically adjusted to account for measured pre-program differences among the 
subgroups in characteristics other than that used to define the subgroups. For example, it presents 
results for ethnic subgroups controlling for differences in measured characteristics other than 
ethnicity, such as site and age. This analysis does control for site differences and thus can be used 
for subgroups that are relatively concentrated in a few sites. 


C. Sample Differences for Key Site Groupings 


In view of the current interest among policymakers in the influence of a site's delivery system 
for education and training — whether it is concurrent, sequential/in-house, or sequential/orokered — 
Table 2.2 collapses the 13 site columns of Table 2.1 into three columns, one for each type of delivery 
system.°° The column labeled "p" is a measure of the likelihood that the observed differences in 


27(.. continued) 
above 2:1 at the Phoenix Job Corps and Allentown in Buffalo. However, Conneliey in Pittsburgh and CREC 
in Hartford were notable exceptions, with substantial majorities of women having had prior-year work 
experience. Men's prior employment profiles by site were the opposite, with ratios of employed to 
nonemployed as high as 4:1 at SER/Corpus Christi, CREC in Hartford, and the Atlanta Job Corps. Only at 
Los Angeles Job Corps and BSA in New York City did male nonworkers outnumber workers, with the 
former’s ratio of nonworkers to workers exceeding 2:1. 
average baseline reading level on entering the program also varied among the sites. This is not 
discussed in detail because comparable initial test scores are not available for all the sites, as outlined earlier. 
2*Defining subgroups based on pre-random assignment characteristics is necessary to maintain the 
legitimacy of the comparisons of experimentals and controls. For example, those who had not worked in the 
year prior to random assignment were just as likely to be randomly assigned to the experimental group as to 
the control group, making comparisons of experimentals and controls with this characteristic valid. 
eight concurrent sites in Table 2.1 are collapsed into the “concurrent” column of Table 2.2; El 
Centro in Dallas and the Los Angeles Job Corps into the "sequential/in-house” column; and Allentown in 
(continued...) 
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TABLE 2.2 
CHARACTERISTICS AT THE TIME OF RANDOM ASSIGNMENT, BY PROGRAM STRUCTURE 


Sequential/ § Sequential/ All 
Characteristic Sample Concurrent In-House Brokered Categories 
and Subgroup Size (%) (%) (%) (%) p® 
Gender 
Women 1,041 51.8 56.3 56.7 53.6 0.125 
Men 900 48.2 43.7 43.3 46.4 
Ethnicity 
White, non-Hispanic 172 10.6 49 7.7 8.9*** 0.000 
Black, non-Hispanic 860 33.2 57.6 66.1 44.3 
Hispanic 847 53.6 30.5 25.6 43.6 
Other 62 26 7.1 0.6 3.2 
Ethnicity, by gender 
White, non-Hispanic 97 5.8 29 4.8 §.0°¢* 0.000 
Black, non-Hispanic 467 18.5 30.5 35.3 24.1 
Hispanic 451 26.6 19.5 16.2 23.2 
Other 26 0.9 3.4 0.3 1.3 
Men 
White, non-Hispanic 75 48 2.0 28 3.9 
Black, non-Hispanic 393 14.7 27.1 20.8 20.2 
Hispanic 396 26.9 11.0 9.4 20.4 
Other 36 1.7 3.7 03 1.9 
Parental status 
Women living with own 
child(ren) 
No §33 26.9 26.8 29.9 27.5*¢ 0.027 
Yes 508 24.8 29.5 26.8 26.2 
Men who have own child(ren) 
No 785 41.4 41.0 36.8 40.4 
Yes 115 69 27 6.6 5.9 
Employed within past year 
914 40.5 62.2 51.6 47.1¢°* 0.000 
Yes 1,027 59.5 37.8 48.4 52.9 
Prior employment, by gender 
Women employed within 
past year 
No 583 26.7 37.3 32.8 30.0°** 0.000 
Yes 458 25.1 19.0 23.9 23.6 
Men employed within past year 
No 331 13.8 24.9 18.8 17.1 
Yes 569 4 18.8 24.5 29.3 
Sample size 1,941 1,180 410 351 
(continued) 
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TABLE 2.2 (continued) 


Sequential/ | Sequential/ All 
Characteristic Sample Concurrent In-House Brokered Categories 
and Subgroup Size (%) (%) (%) (%) p® 
Left school in grade 11 or 12 
No 1,140 60.3 52.9 60.1 §8.7°* 0.027 
Yes 801 39.7 47.1 39.9 413 
Received occupational training 
within past year 
No 1,615 81.4 90.2 80.9 83,2¢¢¢ 0.000 
Yes 326 18.6 9.8 19.1 16.8 
16-19 1,425 70.9 79.0 75.2 73.48*8 0.004 
20 or 21 516 29.1 21.0 24.8 26.6 
Marital status 
Ever married 184 11.8 76 4.0 9.5*** 0.000 
Never married 1,757 88.2 92.4 96.0 90.5 
Living in own household or 
with boy/girlfriend 
No 1,575 80.0 88.5 76.4 81.19" 0.000 
Yes 366 20.0 11.5 23.6 18.9 
Own AFDC case or receiving 
General Assistance 
No 1,418 74.7 75.1 65.0 73.19*8 0.001 
Yes §23 25.3 24.9 35.0 26.9 
Own AFDC case 
No 1,522 80.4 76.1 74.4 78.4*%* 0.023 
Yes 419 19.6 23.9 25.6 21.6 
Receiving Food Stamps 
1,214 61.9 76.6 48.4 62.5°** 0.000 
Yes 727 38.1 23.4 51.6 37.5 
Arrested since age 16 
No 1,649 82.9 89.8 86.3 85.0°** 0.003 
Yes 292 17.1 10.2 13.7 15.0 
Arrested since age 16, 
by gender 
Women 
No 966 48.4 55.2 53.8 50.8°* 0.019 
Yes 55 3.4 1.2 29 28 
Men 
No 663 34.5 34.6 32.5 34.2 
Yes 237 13.7 9.0 10.8 12.2 
Sample size 1,941 1,180 410 351 
(continued) 
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TABLE 2.2 (continued) 


Sequential/  Sequential/ All 
Characteristic Sample Concurrent In-House Brokered Categories 
and Subgroup Size (%) (%) (%) (%) p* 
Lived with both parents at 
age 14 
No 1,264 59.2 72.4 76.6 65.1*¢* 0.000 
Yes 677 40.8 27.6 23.4 34.9 
Reason for leaving regular 
high school 
School-related 925 48.3 42.4 51.6 47,7008 0.002 
Jeb-related 197 11.2 11.5 5.1 10.1 
Other 819 40.5 46.1 43.3 42.2 
She 
Concurrem 
Atlanta Job Corps 69 5.8 0.0 0.0 3.6*e* 0.000 
CET/San Jose 167 14.2 0.0 0.0 86 
Chicago Commons 75 6.4 0.0 0.0 3.9 
(Pittsburgh) 184 15.6 0.0 0.0 9.5 
East LA Skis Center 106 9.0 0.0 0.0 55 
EGOS (Denver) 198 16.8 0.0 0.0 10.2 
Phoenix Job Corps 134 11.4 0.0 0.0 69 
SER/Corpus Chiisti 247 20.9 0.0 0.0 12.7 
El Centro (Dallas) 179 0.0 43.7 0.0 9.2 
LA Job Corps 231 0.0 56.3 0.0 11.9 
Allertown (Buffalo) 135 0.0 0.0 38.5 7.0 
(NYC) 117 0.0 0.0 33.3 6.0 
CREC (Hartford) 99 0.0 0.0 28.2 §.1 
Sample size 1,941 1,180 410 351 


SOURCE: MDRC calculations from JOBSTART enroliment form and survey data. 


NOTES: Calculations for this table used data for all 1,941 sample members for whom there were 48 months of 
follow-up survey data. Sample sizes reported may tall short of this number because of tems missing from some sample 
members’ questionnaires. 


Distributions may not tota' 100.0 percent because of rounding. 
®The column labeled *p’ is the statist'cal significance level of differences among groups in distributions of 


only because of random error. A Pearson chi-square statistic was used to test the hypothesis of equal distributions. 
Statistical significance levels are indicated as °** = 1 percent; ** = 5 percent; * = 10 percent. 
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characteristics among the sites occurred because of random error; the lower the p-value, the more 


likely it is that the observed differences are “real” rather than caused because the sample drawn was 
unusual. Sample members at sequential/in-house sites were younger, less likely to have acknowledged 
fatherhood on the enrollment form, less likely to have worked during the prior year, more likely to 
have completed the 10th grade, and more likely to have previously attended an occupational training 
program than were sample members in other sites. Sample members in concurrent sites were older, 
much less likely to be black, much more likely to be Hispanic, more likely to have worked during the 
prior year, less likely never to have been married, less likely to be receiving AFDC, and more likely 
to have lived with two parents at age 14 than were sample members in other sites.?! One possible 
generalization is that those recruited at concurrent sites had more prior work experience, while those 
at sequential sites had more formal education and training. These differences may be related to the 
sites’ programs, with concurrent sites emphasizing the job connection more and sequential sites 
beginning with education. 


D. Gender Differences in Baseline Characteristics 


An important question in evaluating JOBSTART is whether participation and program impacts 
vary by gender. A first step toward understanding gender differences is to examine the other 
characteristics of the various gender-defined subgroups. Table 2.3 shows that men and women in the 
impact sample were similar in many characteristics, including age, ethnicity, educational attainment, 
and initial reading level. However, men were more likely to have had recent work experience and 
vocational training and to have been arrested since age 16. They were less likely to have been 
married, to be a parent, and to be receiving public assistance. 

Most of these differences between men and women are due mainly to differences between 
custodial mothers and other members of the sample. Custodial mothers were least likely to have 
worked in the year before random assignment, most likely to have lived on their own, most likely to 
have received AFDC and Food Stamps, and least likely to have lived with both parents at age 14. 
Because of these clear differences of custodial mothers in baseline characteristics, the gender-based 


39(...continued) 

Buffalo, BSA in New York City, snd CREC in Hartford into the “sequential/orokered” column. Averaging 
data for sites in broad categories destroys much of the observed site variation — particularly regarding 
ethnicity, receipt of welfare and Food Stamps, parenting status, amount of schooling, and prior-year 
em t, 

3!There may also be unobserved differences. For example, youths attracted to sequential/brokered 
programs run by community-based educational institutions may have been more interested in passing the GED 
examination than youths at concurrent sites run by training agencies. 
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TABLE 2.3 


CHARACTERISTICS AT THE TIME OF RANDOM ASSIGNMEN(, BY GENDER AND PARENTAL STATUS 


Child(ren) Child(ren) Categories 
(%) (%) 


26 8.9*** 
55.7 44.3 
40.9 43.6 

32 


5.0*** 

24.1 

23.2 

1.3 

White, non-Hispanic 3.9 

Black, non-Hispanic K i ; 20.2 
Hispanic I ! . A 20.4 

Other 1.9 
Employed within past year ieiies 


Men employed within ear 
No one 331 0.0 0.0 38.3 26.1 17.1 
Yes 569 0.0 0.0 61.7 73.9 29.3 
Sample size 1,941 508 533 765 115 
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Characteristic and Subgroup 


Left school in grade 11 of 12 
No 


Yes 

Received occupational training 

within past year 
No 1,615 87.8 85.2 79.7 77.4 83.2°*¢ 0.000 
Yes 326 12.2 148 20.3 22.6 16.8 
16-19 1,425 61.2 84.8 775 47.0 73,4%e* 0.000 
20 of 21 516 38.8 15.2 22.5 §3.0 26.6 

Marttal status 
Ever married 184 19.5 58 29 27.0 9.5*** 0.000 
Never married 1,757 80.5 94.2 97.1 73.0 90.5 

Living In own household or 

with boy/girifriend 
No 1,575 §3.9 87.6 94.5 80.0 81.1¢¢* 0.000 
Yes 366 46.1 12.4 55 20.0 18.9 

Own AFDC case or receiving 

General Assistance 
No 1,418 37.2 84.2 87.5 80.9 73.1004 0.000 
Yes §23 62.8 15.8 12.5 19.1 26.9 

Own AFOC case 
No 1,522 39.4 90.1 94.1 89.6 76.4¢*° 0.000 
Yes 419 60.6 9.9 5.9 10.4 21.6 

_ ead 

Sample size 1,941 508 533 7685 115 
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TABLE 2.3 (continued) 


Have Own All 
Others Child(ren) Child(ren) Categories 
Characteristic and Subgroup (%) (%) (%) (%) 


67.7 71.5 60.9 62.5°** 
32.3 28.5 39.1 37.5 


Arrested since age 16 
No 


: 93.8 74.1 70.4 85.0°** 
Yes ' 6.2 25.9 29.6 15.0 


Arrested since age 16, 


Lived with both parents at 
age 14 

No 

Yes 


Reason for leaving regular 


School-related 925 21.5 56.7 59.0 44.3 47.7°** 0.000 
Job-related 197 26 7.3 15.3 21.7 10.1 
Other 819 76.0 36.0 25.7 33.9 42.2 


Sample size 1,941 508 §33 785 115 


(continued) 
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TABLE 2.3 (continued) 


Men 
Do Not 
porate Own Have Own All 
(ren) Child(ren) Categories 
Characteristic and Subgroup (%) (%) (%) 


MORC calculations from JOBSTART enroliment form and survey data. 


NOTES: Calculations for this table used data for all 1,941 sample members for whom there were 48 months of follow-up survey data. Sample sizes 
reported may fall short of this number because of tems missing from some sample members’ questionnaires. 

Distributions may not tote! 100.0 percent because of rounding. 

®The column labeled ‘p* is the statistical significance level of differences among groups in distributions of characteristics: that is, p is the probability that 
observed proportions in each subgroup differ from one column to another only because of random error. A Pearson chi-square statistic was used to test the 
hypothesis of equal distributions. Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; * = 10 percent. 
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subgroup analysis in this report includes three subgroups: men, custodial mothers, and all other 


women. In most of the analysis, all men are usually grouped together because of the small number 


reporting that they were parents at baseline. 


CHIAPTER 3 


JOBSTART SERVICES AND PROGRAM PARTICIPATION 


The JOBSTART model required sites to operate basic education and occupational skills training 
classes that would be interesting and accessible, effective in improving the skills of young people, and 
of relatively long duration. It also required young people to take advantage of these opportunities. 
Historically, education and training programs have had problems retaining young, economically 
disadvantaged dropouts (or even high school graduates).! Thus, a key question for the evaluation 
is whether youths offered JOBSTART services did actually participate in lengthy, intensive services. 
As will become apparent from the findings in this chapter, JOBSTART was not the same program 
for all youths in the experimental group. 

This chapter looks at the JOBSTART experience from three perspectives.? First, it briefly 
summarizes the nature of program services (highlighting key aspects of site variation) and reports 
youths’ subjective reactions to the services.> Second, it describes the participation patterns of youths 
who were active in the JOBSTART Demonstration and compares those patterns to other programs 
for young school dropouts. The analysis shows that participation was, in general, longer and more 
substantial than in most other JTPA Title IIA-funded activities for young dropouts, and that it was 
roughly comparable to participation in intensive programs such as the nonresidential Job Corps and 
the National Supported Work Demonstration (generally referred to simply as Supported Work). 

Third, the chapter analyzes the extent to which participation varied among different groups of 
youths and types of sites. This analysis finds that average participation hours were similar for many 
groups: males and females, various ethnic groups, older and younger participants, youths with 
relatively higher and lower levels of reading skills, and recipients and nonrecipients of public 
assistance. But behind these averages there is substantial variation, with over one-third of the sample 
receiving a very weak version of the JOBSTART program and about one-fifth participating more than 
700 hours. 


1U,S. Department of Education, National Center for Education Statistics, 1989; Public/Private Ventures, 
1988; Kelly, 1987. 
chapter summarizes and updates information presented in Chapters 2 and 4 through 8 of Auspos et 
al., 1989. See that report for more details. 
Sthese reactions were captured in the initial follow-up survey, which was conducted 12 months after 
random assignment, and in focus groups with participants. This section presents information on the 
JOBSTART components in specific sites. 
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The discussion of youths’ experiences reinforces two basic themes of this report. First, the 
variation in the details of the programs highlights the diversity of JOBSTART experiences among the 
sites within the general framework of the JOBSTART guidclines. And second, the experience of the 
sites shows that the basic program model can be implemented in a variety of administrative and labor 
market settings and using different basic program structures, though there were clearly stronger and 
weaker programs among the sites in the demonstration. 


I. The Nature of the JOBSTART Services 


Basic education, occupational training, support services, and job placement assistance were 
available to participants in each site. To operate JOBSTART, two of the six community-based 
organizations (SER/Corpus Christi and Chicago Commons) added education to their regular service 
offerings, and three of the others (Allentown in Buffalo, BSA in New York City, and CREC in 
Hartford) developed or strengthened relationships with outside training programs so that they could 
serve as brokers, arranging training elsewhere for JOBSTART participants. The one community 
college (E! Ceniro in Dallas) and three adult vocational schools (Connelley in Pittsburgh, the East 
Los Angeles Skills Center, and EGOS in Denver) had previously offered education and training but 
had to strengthen support services and job placement assistance. The three Job Corps Centers (in 
Atlanta, Los Angeles, and Phoenix) already had all four kinds of services in place. CET/San Jose 
already operated a program of integrated training and education, with support services and job 
placement assistance. 

Table 3.1, which groups the sites by whether they operated concurrent, sequential/in-house, or 
sequential/orokered programs, describes the entry and exit rules, availability of separate classes for 
youths, expected duration of occupational training, and scheduled hours per day in each site.* In 
some sites, participants could enter courses at any time (open entry) and leave them when they had 
achieved a certain competency level (open exit), while in others they had to adhere to a fixed cycle, 
with entry on specified dates and exit after a set period of time. Some sites held classes for youths 
only, while others mixed youths and adults. Sites also varied in their expected duration of training, 
daily scheduling, and support services. 


“This grouping of sites was chosen because, as discussed later, participation rates by component, 
participation hours, and program emphasis differed among the three types. 
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Chicago Commons 


Connetiey (Pittsburgh) 


East LA Skits Center 


EGOS (Denver) 


Phoenix Job Corps 


SEFYCorpus Christi 


re eee eee eee 


Fixed cycle 


600-840 hours 
Ouring 20.28 
weeks 


in education only 600-1,000 hours 


Yes 


\yew 
maximum 


800-660 hours 


Ouring 22-23 
weeks 


2.8 hours for first 
12-16 weeks® 


12-16 weeks, 
then 6 hours 


6.5-8 hours 


6 hours in school 
year 1965-86, 7 
hours In school 
yeer 1986-87 


6 hours 


6 hours 


6.5 hours 


(continued) 


She tor Training Education® Training Other Activities Total 


Sequential/in-house 


El Centro (Daiias) 2-3 hours In life 6 hours 
24 weeks activities 


Open entry and = in education only Varied by training 3 hours 6 hours during 
ext for provider education phase 
education, varied 
in training 
Open entry and = in education only Varied by training 3 hours, 4 days 6 hours during 
ext for provider per week provider education phase, 
education, 4 days per week 
in training 

CREC (Hartford) Open entry and = No Varied by training 3 hours Vatied by training None” 3 hours during 
exit for provider provider education phase 
education, varied 


ee ee a a ee 
SOURCE: Adapted from Auspos et al., 1989. 
NOTES: Education hours refer to time spent in a basic education or GED-preparation class and do not include education provided as part of an occupational 

‘aining course. 
Job Corps Centers offered a maximum of two years of training, but JOBSTART participants were supposed to be enrolled in courses that could be completed in 
‘Some participants had paid or unpaid work experience positions for limited periods. 
SL ve skits classes typically provided instruction in work behaviors, goal-setting, personal budgeting, health, and interpersonal relations. Avocationa! activities 
education. 


Included physical education and driver 
®Additional hours were available on an individualized basis afier the course ended. 127 
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A. Basic Education 


The education component typically consisted of individualized instruction, which allowed 
students to move at their own pace learning reading, mathematics, and other subjects needed to pass 
the GED examination. Mostly they worked on their own, doing workbook exercises or, less 
commonly, using computer-assisted instruction. In sites offering education and training concurrently, 
participants usually attended two hours of education classes and four hours of vocational training a 
day. In sites operating a sequential program, participants generally attended three hours a day of 
basic skills classes during the education phase, with the remaining three hours a day devoted to life 
skills classes. 

The payment provisions of the contracts between service providers and funding agencies 
(especially local SDAs) were an important source of variation in the emphasis of the education 
component. In four sites (Connelley in Pittsburgh, EGOS in Denver, El Centro in Dalles, and 
SER/Corpus Christi), payment for education services was based on students passing the GED 
examination. This led these sites to make GED certification an important short-term goal of the 
program and to emphasize the skills tested on the GED examination in their education component. 
Other sites —- CET/San Jose, the East Los Angeles Skills Center, and especially Chicago Commons 
- saw GED attainment as a long-term goal and did not stress it in their JOBSTART programs, 
focusing more on improving basic skills as an aid to vocational training and job placement. 

The actual curricula and instructional materials were not specified by the JOBSTART 
guidelines. The three Job Corps sites used the standard Job Corps materials (workbooks, textbooks, 
and audiovisual materials), though two centers (Atlanta and Phoenix) also had supplementary 
computer-assisted instruction. The three sequential/brokered sites used the Comprehensive 
Competencies Program (CCP) developed by U.S. Basic Skills Investment Corporation. CCP is an 
instructional management system integrating textbooks, workbooks, computer software, audiovisual 
materials, and progress tests. In the seven other sites, teachers developed their own instructional 
materials using a variety of sources, such as GED preparation courses and reading and mathematics 
textbooks that use the “mastery learning” approach, which focuses on the step-by-step acquisition of 
specific competencies. In four of these sites, staff supplemented pencil-and-paper exercises with 
computer-assisted instruction.5 

Teachers in most sites felt that the individualized, self-paced instruction provided a better 


SSee Auspos et al., 1989, for the details of these programs. 
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learning environment than participants had typically found in high school. The competency-based 
courses allowed the youths to see themselves making incremental progress as they advanced toward 
what was, for many, a remote goal of mastering basic skills and receiving a GED. Most students 
preferred this instructional approach because they felt that it made them active participants in the 
process of learning and allowed them to master one topic before beginning another. In the follow-up 
survey, about three-fourths of JOBSTART participants found self-paced instruction “very helpful," 
while virtually no one found it “not helpful at all.” 

Yet students also valued interaction with instructors, as much for the personal attention and 
motivation it provided as for instruction in specific skills. About 75 percent of JOBSTART 
participants rated support from teachers and fellow students in the education component “very 
helpful." 

Despite the overall favorable assessment, three concerns emerged. First, with a few important 
exceptions, the basic education and skills training activities operated separately, with little integration 
of material. As discussed in Chapter 1, only at CET/San Jose, and to a lesser extent Chicago 
Commons, were basic skills and occupational training instruction truly integrated. Though several 
other sites did attempt to coordinate the two activities to a limited extent (creating a distribution of 
sites rather than two clear-cut categories), these sites fell short of the integration observed at CET/ 
San Jose and Chicago Commons. Second, some instructors thought the curriculum should include 
more material on critical thinking and general knowledge, in contrast to the functional literacy and 
mathematics emphasis of many integrated programs. Third, some instructors said that students with 
very poor skills or low motivation found the work boring and, as a remedy, suggested more group 
activities. One site, El Centro in Dallas, shifted to this approach, relying more heavily than other 
sites on class exercises and lectures. 


B. Occupational Skills Training. 


The choices of occupational training available to participants varied among the sites. 
Participants at large vocational schools could choose courses in more than 20 occupational areas. The 
Job Corps Centers and larger community-based organizations also offered a wide range of vocational 
training. In contrast, SER/Corpus Christi, which provided training in-house, offered only a few 
courses. 

In theory, youths in sequential/orokered sites could choose courses from a variety of local 
agencies. However, in practice, some courses were unavailable to them because they could not satisfy 
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entrance requirements, or other difficulties prevented them from gaining entry.° As discussed later 
in this chapter, the resulting low rate of participation in training in sequential/brokered sites was the 
major operational issue concerning the training component. 

As a group, JOBSTART participants were enrolled in training for a broad range of occupations 
— clerical and service jobs, machine trades, benchwork occupations, and structural work such as 
welding. Occupational choices for men and women followed traditional patterns, as shown in Table 
3.2, with about three-fourths of the female participants training for clerical jobs.’ 

Using categories employed by the U.S. General Accounting Office (GAO) in a recent analysis 
of JTPA Title IIA adult training, MDRC classified the JOBSTART training provided to participants 
as leading to jobs requiring low or low/moderate skills (slightly less than one-fourth of participants), 
moderate skills (about one-half of participants), and higher skills (about one-fourth of participants).® 
This distribution of skills ratings for training occupations was similar to what GAO found for JTPA 
Title ILA adult programs. This was unexpected, since JOBSTART participants faced more barriers 
to employment than did the typical JTPA Title IIA adult client. 

One argument for sequential programs is that the upfront education allows participants to enter 
more advanced training. In the JOBSTART Demonstration, however, this did not appear to occur. 
In terms of the GAO categories, the jobs for which youths trained in sequential sites did not appear 
to require higher skills than those in concurrent sites. 


C. Support Services to Facilitate Participation 


All sites provided basic support services such as assistance with transportation and child care, 
which helped participants to attend the program, as shown in Table 3.3. All sites provided bus passes 
or small allowances to cover the costs of commuting to the program. JOBSTART counselor/ 
coordinators placed a high priority on adequate child care arrangements. In most sites, staff referred 
JOBSTART participants to other agencies for child care, with the expenses being covered by JTPA 
or the Work Incentive (WIN) Program. The Atlanta Job Corps, two CBOs (SER/Corpus Christi and 
CET/San Jose), and one adult school (Connelley in Pittsburgh) had on-site day care facilities, but staff 


Sas mentioned in Chapter 1, the JTPA performance standards created an incentive for SDAs to emphasize 
lower-cost, short-term programs. Some SDAs in study sites were reluctant to provide a single individual with 
both education aud training, and many JTPA Title ILA-funded service providers operated under performance- 
based contracts linking payment to placement in a job. Both practices hindered the efforts of JOBSTART 
youths in sequential/orokered sites to find a training agency willing to accept them. 

*This table, taken from an earlier report on JOBSTART (Auspos et al., 1989), is based on a similar, but 
slightly smaller, sample than that used for this 

®See U.S. General Accounting Office, 1988, for the definitions of categories of training. The percentage 
distribution reported above for JOBSTART was calculated in Auspos et al., 1989, based on a slightly different 
sample of participants than that used in this report. 
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TABLE 3.2 


PERCENTAGE DISTRIBUTION OF OCCUPATIONS 
FOR PARTICIPANTS IN JOBSTART TRAINING, BY GENDER 


Men Women Men and Women 
Training Category* (%) (%) (%) 
Clerical and sales occupations 
Stenography, typing, filing, 
and related occupations 5.0 51.3 29.2 
Computing and account-recording 7.0 20.3 13.9 
Production and stock clerks, 
and related occupations 0.3 0.0 0.1 
Information and message distribution 1.4 0.0 0.7 
Miscellaneous clerical 0.0 0.5 03 
Sales and consumable commodities 0.8 1.0 0.9 
Total 14.6 73.1 45.1 
Service occupations 
Food and beverage preparation and services 3.1 4.6 3.9 
Miscellaneous personal services 0.6 11.5 6.3 
Building and related services 8.4 21 5.1 
Total 12.0 18.2 15.3 
Machine trades occupations 
Metal machining 5.0 0.8 2.8 
Mechanics and machinery repair 22.4 1.3 11.4 
Printing 0.6 1.3 0.9 
Wood machining 0.6 0.5 0.5 
Total 28.6 3.8 15.7 
Benchwork occupations 
Assembly and repair of electrical equipment 11.5 1.3 6.2 
Painting, decorating, and related occupations 0.8 0.3 0.5 
Fabrication and repair of plastics, synthetics, 
rubber, and related products 2.2 0.3 1.2 
Fabrication and repair of textile, leather, 
and related products 1.7 0.3 0.9 
Total 16.2 21 8.8 
Structural work occupations 
Metal 9.8 0.5 5.0 
Welders, cutters, and related occupations 0.8 0.0 0.4 
Electrical assembling, installing, and repairing 5.9 0.5 3.1 
Painting, plastering, waterproofing, 
cementing, and related occupations 17 0.0 0.8 
Construction 8.4 1.3 47 
Total 26.6 23 13.9 
Miscellaneous occupations 
T 0.0 0.3 0.1 
Graphic art work 2.0 0.3 1.3 
Total 2.0 0.5 1.2 
All training categories 100.0 100.0 100.0 
Sample size 357 390 747 
(continued) 


TABLE 3.2 (continued) 


SOURCE: Adapted from Auspos et al., 1989. The categorization of occupations is derived 
from U.S. Department of Labor, 1977. 


NOTES: Calculations for this table used data for all experimentais who were active for at 
least one hour in a JOBSTART training component within 12 months of random assignment and 
responded to the 12-month follow-up survey. 

Distributions may not total 100.0 percent because of rounding. 

When totaled, individual category percentages may not equal the general category 


because of rounding. 
Tests of statistical significance were not performed. 
“individuals participating in more than cne training category were included in the category in 
which they attended the most hours. 
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Phoenix Job Corps 


BASIC SUPPORT SERVICES AVAILABLE IN JOBSTART, BY SITE 


Needs-Based Payments 


Basic allowance of $40 per Bus passes On-she 


month for first 2 months, $60 
for next 3 months, $80 after 
5 momhs 


Basic allowance of $40 per Bus passes 


month for first 2 months, $60 
for next 3 months, $80 efter 
5 months 


Basic allowance of $40 per 

month for first 2 months, $60 
for next 3 months, $80 after 

§ months 


$5 per day® 


Transportation Child Care 


$2 per day or bus On-site and by 
eS referral 


Other 


Free meals; clothing allowance 
Of $75 In first month, $50 in 


"third month, $96 in sith and 


tenth months, $51 In twelfth 
month; on-site medical and 
dental care 


Free meals; clothing allowance 
Of $75 In first month, $50 in 
third month, $96 in sixth and 
tenth months, $51 in twelfth 
month; on-site medical and 
dental care 


Free meals; clothing allowance 
of $75 In first month, $50 in 
third month, $96 In sixth and 
tenth months, $51 in twelfth 
month; on-site medical and 
dental care 


$50 one-time clothing grant 


Incentive Payments 


Mert raises could increase 
basic allowance to $100 per 
month efter 6 mo:iths; $75 per 


$100 per month after 6 
months; $150 bonus in tenth 
month 


Mert raises could increase 
basic allowance to $100 per 
month after 6 months; $75 per 
month was placed in escrow 
for enrollees who stayed 6 
months, which increased to 
$100 per month after 6 
months; $150 bonus in tenth 
month 


Merit raises could increase 
basic allowance to $100 per 
month after 6 months; $75 per 
month was placed in escrow 
for enrollees who stayed 6 
months, which increased to 
$100 per month after 6 
months; $150 bonus in tenth 
month 


$50 for passing GEO: $50 for 


Ste Needs-Based Paymens 
East LA Skits Center None 
EGOS (Denver) None 
El Centro (Dalles) $5 per day 
Community-based 
organizations 
Allemtown (Buffalo) $1 per hour # on AFOC, 


BSA (NYC) $23-$30 per week during 
education,© $30 per week 
during JTPA training 

b 

- CET/San Jose $1 per hour, for tarmworkers 
only 

Chicago Commons $6 per day 
CREC (Hartiord) None 
SER/Corpus Crvisti $8 per day 


SOURCE: Adapted from Auspos et al., 1969. 


Transportation Chiid Care Other incentive Payments 

Bus passes, By ceferral Emergency funds, lunch money None 

gasoline vouchers during a brie? period 

Bus passes, By referral Lunch money during a brief None 

gasoline vouchers period 

Bus passes By referral Emergency rem funds $5 per week for perfect 
attendance 

included in needs- By referral None None 

based payment 

included In needs- By referral, $15 per Free breakfasts $5 for weekly 


based paymem; = week for expenses 


Bus passes for On-site and by Weekly food bank to provide 
free groceries 


others who 

Gemonstrated need 

Included In needs- By referral None 
based payment 

Bus passes By referral None 


Included in needs- On-site for children None 
based peyment — over 18 months and 
by referral 


TES: "At intervals, this site combined transportation and needs-based payments into one $7 per day payment. 


Available during 1986-87 schoo! year. 
Curing October 1986-August 1967. 
Savaitiable after October 1987. 


academic 
Progress; $§ for perfect weekly 
attendance’ 


None 


GED test; $45 for °A’ average 
training, $25 for °B* average 
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reported that students frequently preferred to make their own arrangements in their own 
neighborhoods. 

Many sites also found ways to provide small payments to meet other costs of participating, 
though the Job Corps Centers were consistently able to provide more support than the other sites. 
Ten of the 13 sites were able to provide some type of small needs-based payment, while nine of the 
sites provided on-site meals or food to take home, or special allowances for clothing or to meet rent 
emergencies. Seven of the sites (including all three Job Corps Centers) provided some form of 
incentive payments to participants who reached milestones in the program. The Job Corps Centers 
also provided on-site medical and dental care. 

In addition, to increase participants’ motivation and commitment to the program, site staff used 
a variety of strategies: personal counseling, peer support, time management training, and group 
recreational activities. Finally, staff at most sites provided training in life skills — covering topics such 
as health, personal finances, and workplace routines — to help the young people function more 
responsibly and productively in a variety of roles and situations. Six of the sites (the three Job Corps 
programs, E! Centro in Dallas, Allentown in Buffalo, and BSA in New York City) incorporated two 
to three hours of formal life skills classes into the regular program day.” The remaining seven sites 
did not focus as systematically on life skills, instead incorporating these topics into the training 
curriculum, counseling or group discussion sessions, or occasional lectures. 

Youths cited personal attention from staff as a crucial aid in helping them move toward 
self-sufficiency. While agencies that traditionally served disadvantaged youths typically offered these 
support services from the beginning of the demonstration, a number of sites accustomed to serving 
adults increased this type of activity as their programs evolved. 

Clearly, the support services and other activities available at the Job Corps Centers surpassed 
those at the other sites in both breadth and intensity. To a large extent, the JOBSTART participants 
in these sites were able to partake of the full array of Job Corps activities, including recreational, 
health, and food services. However, Allentown in Buffalo, Connelley in Pittsburgh, and SER/Corpus 
Christi also offered high leveis of these services. 


D. Job Placement Services 


Sites were required to assist youths in finding training-related employment, but this phase of 
the program typically received less attention than others. Nearly all the sites did provide instruction 


*The life skills curricula in these sites were oriented around daily living, with units on health education, 
substance abuse prevention, sexuality and family planning, personal finances, civics, communication skills, goal- 
setting and planning, and improving self-esteem. 
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about employers’ expectations as well as job search techniques. About one-half of the sites arranged 
paid or unpaid part-time work experience positions for some participants during the program. 


Approximately one-fourth of a sample of participants worked at some point — in program-arranged 
or self-initiated jobs — while they were active in the program. Those in the sample who were 
employed worked an average of about 50 percent of the weeks they were in the JOBSTART program 
and were employed an average of about 30 hours per week during the weeks they worked. During 
the months they worked, their hours of classes in JOBSTART were lower than were those of 
nonworking participants. 

Efforts to find participants permanent employment typically began near the end of training, with 
instructor contacts serving as an important source of information about job openings. Since many 
youths left the program without reaching this stage, it is not surprising that only .bout one-fourth of 
participants reported that program staff referred them to a job or told them about openings. 

Job placement assistance was especially strong at CET/San Jose, Chicago Commons, and the 
Job Corps program in Phoenix. In all these sites, instruction in proper work behavior, employer 
expectations, and job search techniques began while students were still in training; placement 
specialists provided leads and assistance in | nding a job; and CET/San Jose and Chicago Commons 
had especially strong ties to local employers. 

Job placement assistance was noticeably weaker at Allentown in Buffalo, BSA in New York 
City, and CREC in H::tford (the three sequential/orokered sites), and at the Atlanta Job Corps, the 
East Los Angeles Skills Center, EGOS in Denver, and SER/Corpus Christi (among the concurrent 
sites). At the sequential/brokered sites, job placement was intended to be the responsibility of the 
training agency, but most participants were never active in that component, so only informal 
assistance was available from the JOBSTART agency. Those concurrent sites with weak job 
place.nent typically lacked any or sufficient in-house job development specialists, were larger agencies 
with no special emphasis on placing JOBSTART youths, or contracted out job placement to another 
organization that did not see the JOBSTART youths as a high-priority group. 


Sites also varied in the way they scheduled classes and the expected duration of their programs. 
The demonstration sites scheduled JOBSTART classes in three basic ways. The majority of sites that 
operated both the education and training components themselves scheduled the classes on an open- 
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entry/open-exit basis.!° This means that participants could enter the program at any time, progress 
through the material at their own pace, and complete the course whenever they reached the specified 
competency levels. The duration of training was cpen-ended, but sites anticipated that participants 
would typically be able to complete the prescribed training curriculum in many fields in approximately 
600 to 800 hours. Individuals who needed additional time to complete competencies could stay 
longer, however. 

In a second program variation, some concurrent sites operated JOBSTART as a series of “fixed 
cycles,” meaning that all participants started and completed training together on specified dates and 
the maximum length of training was prescribed!’ In a third variation, the three sequential/ 
brokered sites operated the education component on an open-entry/open-exit schedule, but the 
training schedule was determined by the variety of training organizations at which JOBSTART 
participants were enrolled. 

Sites also showed great variety in the number of hours scheduled for activities each day. The 
usual schedule ranged from a low of three hours per day at CREC in Hartford to seven to eight 
hours per day in some courses at Chicago Commons. A typical day can be described in terms of 
three basic models: 

¢ Concurrent sites that were CBOs or schools. Students typically had six hours of 


classes per day, five days a week. In general, two hours were spent in education 
classes, with training classes scheduled for the remaining four hours. 


¢ Concurrent sites that were Job Cops Centers. These sites had six and a half class 
hours per day. Schedules were highly individualized and changed frequently, but 
commonly included two hours of education, two and a half hours of vocational 
training, and two hours devoted to life skills, health education, or avocational 
activities such as sports. 


* Sequential sites. These also scheduled a six-hour day during the education phase, 
but the daily distribution of activities was quite different. Typically, three hours 
were spent in education classes and another three hours were spent in life skills 
training. The training schedules were set by the training providers at the brokered 
sites, but typically involved five to six hours of classes per day. Training classes ran 
for six hours a day at the sequential/in-house sites. 


The duration of the occupational training component also varied among the sites, ranging from 
22 to 23 weeks at SER/Corpus Christi to a year at the Job Corps sites. Even within a site there could 


1These sites included concurrent sites (the Atlanta Job Corps, CET/San Jose, the East Los Angeles Skills 
Center, and the Phoenix Jobs Corps) and sequential/in-house sites (El Centro in Dallas and the Los Angeles 


Corps). 
MThese sites included Chicago Commons, Connelley in Pittsburgh, EGOS in Denver, and SER/Corpus 
Christi. 
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be significant variation among the different training options. At Chicago Commons, for example, 


scheduled training ranged from 500 hours in industrial inspection to 1,380 hours in packaging-machine 
repair. 

This diversity in scheduled daily hours and program duration meant that the planned 
participation hours for youths varied greatly across the sites, with the greatest variation showing in 
the training component. At SER/Corpus Christi, a participant completing education and training in 
about six months, as planned, would have had no more than 660 hours of occupational training. In 
contrast, one training course at Chicago Commons totaled nearly 1,400 hours, and a sequential 
program such as the one operated by the Los Angeles Job Corps could last for as long as a year.!2 


F. Summary of Program Implementation by Site 


Table 3.4 rates the implementation of the four central JOBSTART components in each site. 
(See Auspos et al., 1989, for the details behind these ratings.) The information in Table 3.4 and the 
material already presented in this section suggest that the four key components of the JOBSTART 
program were implemented most successfully at CET/San Jose,'> Chicago Commons, Connelley in 
Pittsburgh, E] Centro in Dallas, the Los Angeles Job Corps, and the Phoenix Job Corps, and least 
successfully at the Atlanta Job Corps, BSA in New York City, CREC in Hartford, and EGOS in 
Denver. To summarize the ratings by component: 


¢ Education. Most sites that chose to operate a separate education component were 
able to offer an activity that met the JOBSTART guidelines. The two sites with 
noticeably weak education activities were the Atlanta Job Corps (where unclear 

e Objectives for education and staff turnover hampered implementation) and CREC 
in Hartford (where computer facilities were underutilized and attendance was a 
serious problem). 


® Training. The training component showed the most variation — primarily because 
in sequential/orokered sites most youths never participated in training. In addition, 
the limited training offerings, less experienced staff, and older equipment of 
SER/Corpus Christi (reflecting the common problems of a community-based 
organization) hampered its ability to implement the training component. 


¢ Support services. Although the Job Corps Centers did offer substantially more 
services than other sites, all programs were able to provide the limited types of 


12Job Corps Centers offered a maximum of two years of training, but JOBSTART participants were only 
to be enrolled in courses that could be completed in one year. 

ISCET/San Jose provided most of its basic education services within training activities, so this overall 
characterization reflects a judgment about the other three components and the way in which education was 
incorporated into the training component. 
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TABLE 3.4 
RATINGS OF THE IMPLEMENTATION OF JOBSTART COMPONENTS, BY SITE 


Support Job 


Site Education Training Services Placement 
Concurrent 
Atlanta Job Corps Low Medium High Low 
CET/San Jose No rating® High Medium High 
Chicago Commons Medium High Medium High 
Connelley (Pittsburgh) High High High Medium 
East LA Skills Center Medium Medium Medium Low 
EGOS (Denver) Medium Medium Medium Low 
Phoenix Job Corps Medium High High High 
SER/Corpus Christi High Low High Low 
SequentiaVin-house 
El Centro (Dallas) High Medium Medium Medium 
LA Job Corps ledium Medium High Medium 
SequentialVbrokered 
Allentown (Buffalo) High Low High Low 
BSA (NYC) Medium Low Medium Low 
CREC (Hartford) Low Low Mecium Low 


SOURCE: MDRC operations staff. 


NOTES: See Auspos et al., 1989, for details of the implementation of components in each site. 

*In this site, a separate rating of the education component was inappropriate because education 
and training were more integrated than in other sites and staff strongly emphasized training over 
passing the GED examination. 
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support services called for in the JOBSTART guidelines: assistance in arranging 
and/or financing child care and transportation to and from coursework. Allentown 
in Buffalo, Connelley in Pittsburgh, and SER/Corpus Christi, in addition to the 
three Job Corps Centers, provided a noticeably longer list of services, including 
better needs-based payments, life skills training and counseling, and a method of 
identifying service needs and making referrals of youths to other agencies providing 
the required services. 


© Job placement assistance. Most sites offered job placement assistance that fell 
short of the JOBSTART guidelines, either because many youths never received the 
service (especially in sequential/brokered sites) or too few staff with a specialty in 
job search assistance were assigned to work with the JOBSTART youths. As 
mentioned earlier, job placement assistance was especially strong at CET/San Jose, 
Chicago Commons, and the Phoenix Job Corps. 


Il. The Intensity of JOBSTART Participation 


Participation in JOBSTART among experimentals was measured by participation rates in each 
activity, hours of participation in each activity, and overall length of stay. Table 3.5 shows these 
summary measures for all experimentals in the impact sample:!4 


° Participation rates. Nearly 90 percent of all experimentals in the impact sample 
participated in JOBSTART to some extent. Eighty-six percent of all experimentals 
(and nearly all of those who were active in JOBSTART) attended basic skills 
education classes, 67 percent participated in training, and 40 percent participated 
in other activities, which were optional for sites. 


¢ Average participation hours. Average hours were 125 in education, 238 in 
training, and 37 in other activities, for a total of 400 hours.'5 


* Distribution of participation hours. Forty-one percent of all experimentals spent 
fewer than 201 hours in all JOBSTART activities; 25 percent spent 201 to 500 
hours; and 33 percent spent more than 500 hours. 


© Length of stay. The average length of stay was 6.6 months, with the median length 
being slightly less, 5.9 months; 77 percent of experimentals were active for 3 
months or more, while 53 percent stayed in the program for 6 months or more. 
Length of stay was measured from the time of random assignment through the last 


144s discussed in Chapter 2, the findings in this report are slightly different from those presented in prior 
reports because of small differeaces in the samples used for the analysis. 
1SThese averages and those cited in the next paragraph include the 11 percent of the sample with zero 
hours of participation in JOBSTART activities. 
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TABLE 3.5 
PARTICIPATION RATES, HOURS OF PARTICIPATION, AND LENGTH OF STAY 


FOR EXPERIMENTALS 
Activity Measure Outcome 
Percent participating in 
Any activ’ 88.8 
Education 85.5 
Training 66.5 
Education and training 65.2 
Other activities 39.7 
Average hours in 
Education 124.5 
Training 238.2 
Education and training 362.8 
Other activities 36.9 
All activities 399.9 
Percentage distribution of hours 
in education and training 
None 11.9 
Up to 200 35.0 
201 to 500 21.6 
501 to 700 15.9 
701 or more 16.6 
Total 100.0 
Percentage distribution of hours 
in all activities 
None 11.2 
Up to 200 30.0 
201 to 500 25.4 
501 to 700 15.1 
701 or more 18.3 
Total 100.0 
Length of stay (months) 
Average 6.6 
Median 5.9 
Months receiving services 
Average 5.9 
Median 5.0 
Percent still participating in month = 
7 
6 52.9 
9 29.5 
12 15.4 
15 8.7 
18 4.4 
19 or later 3.3 
Sample size 988 


SOURCE: MDRC calculations from JOBSTART enrollment form and MIS data. 


NOTES: Calculations for this table used data for all 988 experimentals for whom 
there were 48 months of follow-up survey data, those with values of zero for 
outcomes and those who were assigned to JOBSTART but did not participate. 

Distributions may not total 100.0 percent because of rounding. 
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month that included any hours of participation.!® Fifteen percent of the exper- 
imental sample were still active in the program in the twelfth month after random 
assignment, while 9 percent were still active in the fifteenth month. 


These findings show that JOBSTART succeeded in engaging more than half of the youths in 
the experimental group in the program and its activities, but that for about 40 percent of them 
participation was quite low and JOBSTART did not constitute an intensive or lengthy program. 
Because of this wide range of participation levels, with some participants receiving very few hours of 
education and training, the average total hours for the sample as a whole is the equivalent of less 
than three and one-half months of regular attendance for six hours per day. Most people in the 
sample did not participate long enough to get a GED or complete a training course. 

To place these results in context, JOBSTART participation may be compared to reported 
participation in other programs for young, disadvantaged school dropouts. Length of participation 
is a simple measure that permits comparison with three types of youth programs: JTPA Title IIA 
programs for young dropouts, the Job Corps, and Supported Work.!” JTPA Title IIA typically 
provides relatively short-term activities, while the Job Corps and Supported Work have been among 
the most intensive employment and training programs for disadvantaged youths. In these 
comparisons, either the average or median length of participation was used, depending on the 
available data. 

Overall, JOBSTART participants stayed in the program considerably longer than did young 
dropouts in JTPA Title IIA activities, as shown in Table 3.6. During program year 1986, when the 
demonstration was in operation, the median length of participation for all young dropouts in JTPA 


164s indicated in Table 3.5, the average number of months during which participants actually received 
services is somewhat lower — 5.9 months. The “length of stay” in the program is higher because the period 
of participation could include months of inactivity if a person stopped attending classes and then returned to 
the program within the 12-month follow-up period. However, this does not appear to have been a common 
pattern. Among a sample of participants, about 85 percent did not have any months of inactivity within the 
period they were counted as active, and among those with inactivity, the average period of inactivity was about 
two months. Youths who attended JOBSTART were counted as participating for the entire month in which 
they were randomly assigned and all months in which they showed any JOBSTART hours. The measure might 
have overestimated the length of participation somewhat when a youth was randomly assigned late in a month 
or ended mel ees tion early in a month. 
For information on the Job Corps, see Richardson and Burghardt, 1985, and U.S. Department of Labor, 
OR le and Training Administration, 1987. On Supported Work, see Maynard, 1980. For JTPA Title 
IIA, see U.S. Department of Labor, 1988. Hours of attendance were not reported for all the programs, hence 
this comparison uses length of stay, for which the data were available. 
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TABLE 3.6 


PARTICIPATION AND LENGTH OF STAY FOR YOUNG DROPOUTS 
IN JTPA TITLE IIA PROGRAMS, BY ACTIVITY 


Percentage Median Length 
Distribution of of Stay 
Activity Youths in JTPA (Months) 
Classroom activities 
Basic education 22.8 3.71 
Occupational skilis training 15.6 3.98 
Combined basic education and 
occupational skills training* 46 6.97 
Total 42.9° 3.97 
On-the-job training 12.2 3.14 
Job search assistance 15.3 0.81 
Work experience 7.8 3.67 
Other services 21.8 3.59 
Any activity 100.0° 3.40 


SOURCE: U.S. Department of Labor, Division of Performance Management 
Evaluation, 1988. . 


NOTES: This table includes data for young dropouts served under JTPA Title IIA 
program year 1986. 

*JTPA data (as recorded by the U.S. Department of Labor, Division of Performance 
Management and Evaluation, 1988) combined basic education and occupational skilis 
training under the label CT-Other. 

©The distribution may not total 42.9 percent because of rounding. 

°The distribution may not total 100.0 percent because of rounding. 


Title ILA programs was 3.4 months compared to 6 months for JOBSTART.'® JOBSTART's median 
length of participation exceeded that of young dropouts in all JTPA components except one. The 
exception was a program combining basic education and occupational skills training, a mix similar to 
JOBSTART's, which had a median length of 7 months but was offered to only 5 percent of all young 
dropouts in JTPA Title IIA activities. For JOBSTART participants active in both education and 
skills training, the median length of stay in the program was also approximately 7 months. These 
findings support the conclusion that JOBSTART achieved its goal of operating a program more 
intensive than that typically offered in JTPA Title IIA programs for young dropouts. JOBSTART 
participation was also longer than that observed for out-ot-school youths in the National JTPA Study, 
where less than ha'f were still enrolled in JTPA three months after random assignment.!? 

JOBSTART's average length of participation was less than that of the Job Corps or Supported 
Work. During program year 1986, the average stay in the Job Corps was 6.9 months, compared to 
JOBSTART's average of 6.6 months.2° Supported Work was an experimental program of paid 
work experience under conditions of gradually increasing responsibility on the job, close supervision, 
and work in association with a crew of peers. It operated from 1975 to 1979 and included young 
school dropouts, many with a criminal record, as one of its target groups. While precise comparisons 
are impossible, the length of participation in Supported Work appears to have been slightly 
longer.2! The average length of participation in Supported Work was 6.7 months (compared to 6.6 
in JOBSTART) and the median was approximately 6 months (the same as in JOBSTART), and 25 
percent of Supported Work participants were still active in the program at 12 months after random 
assignment, as opposed to 15 percent for JOBSTART. 

In summary, while only approximate comparisons can be made, it appears that JOBSTART 
achieved its goal of providing young school dropouts with more intensive education and training than 
is usual within the JTPA system. The data also suggest that JOBSTART offered a duration of 
activity almost as long as that of the Job Corps or Supported Work, which operated through special 
agencies and had the sole mission of providing services to very disadvantaged individuals. However, 


1The average length of participation in JTPA Title ILA programs was not available from published 
sources. The figure for JTPA Title IIA includes only persons who actually participated, while the JOBSTART 
figure includes the 11 percent of the sample made up of nonparticipants with zero months of activity. 

19Bloom et al., 1993. 

2°The median for the Job Corps was not available. 

21The JOBSTART measure of length of participation included some periods of inactivity in the midst of 
participation, while the Supported Work measure factored these out. As discussed above, however, this 
problem does not appear to have been serious in the JOBSTART data. 
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for the 41 percent of the JOBSTART sample who had very few hours of activity or did not 
participate, the treatment was unlikely to be intense and lengthy enough to move them above the 
threshold of skills needed to secure significantly better jobs than they could before the program. 


Ill. Moving Behind the Aggregate Participation Measures 


Aggregate measures tell only part of the story, however. Table 3.5 makes clear that 
JOBSTART was not the same experience for all youths: 41 percent participated for 200 or fewer 
total hours, while more than 18 percent exceeded 700 hours, the required offering under the 
demonstration (200 hours of education and 500 hours of training). Clear differences in average 
participation also existed among the sites, as discussed later in this chapter. Understanding the 
sources of these variations in participation is the first step in developing ways to improve the design 
and implementation of the program. 

The following analysis begins with subgroups of JOBSTART experimentals defined based on 
pre-random assignment characteristics. It shows that while there were differences among subgroups, 
they did not seem to account for all the variation in participation. This implies that factors such as 
unmeasured differences among youths, local employment opportunities, and program characteristics 
associated with particular sites may also have affected participation. The key finding on program 
characteristics is that youths in sites operating sequential/brokered programs tended to have lower 
rates of participation in occupational skills training, although they tended to receive more intensive 
instruction in basic skills. 

A. Differences in Participation Among Subgroups 


Although JOBSTART youths all satisfied the program's eligibility requirements, when they 
entered the program they varied in gender, age, marital and parental status, criminal records, and 
educational attainment, among other characteristics. Research and operational experience suggest 
that these types of factors can influence participation in programs.” 

Among JOBSTART experimentals, two subgroups are of special concern: males (who have 
often been hard to recruit and retain in education and training programs) and young mothers (a 
group at risk of long-term welfare receipt). As Table 3.7 shows, average total hours and other 
measures of participation were similar for all males and females, although a higher percentage of 
females were active in the twelfth month after random assignment. There were some differences, 


2See, for example, Public/Private Ventures, 1988, and Mathematica Policy Research, 1985. 
-79. 
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TABLE 3.7 


PARTICIPATION RATES, HOURS OF PARTICIPATION, AND LENGTH OF STAY, 
BY GENDER AND PARENTAL STATUS 


Living Not Living Men 
with Own with Own All and 
Activity Measure Men Chitd(ren) __Chita(ren)® Women women 
aie ai " 88.4 89.9 88.3 89.1 88.8 
Education 84.8 86.8 85.5 86.1 85.5 
Eg 66.7 68.9 64.0 66.3 66.5 
3 and training 64.7 67.7 63.6 65.6 65.2 
35.7 42.8 43.1 43.0 39.7¢° 
Average hours in 
3 118.3 115. 142.9 129.7¢¢ 124.5 
Lights I 249.4 226.0 231.7 229.0 238.2 
and training 367.7 341.3 374.5 2 362.8 
Other activities 31.5 3 45.3 415 36.9°** 
All activives 399.1 4 419.8 400.6 399.9 
distribution of hours 
in 8nd training 
12.3 10.9 12.4 11.7 19 
to 200 31.9 42.0 33.6 37.6 35.0 
to $00 23.2 17.1 23.0 5 21.6 
501 to 700 18.1 148 13.4 14.2 18.9 
701 or more 14.5 18.2 17.7 16.0 18.6 
Total 
Percemtage distribution of hours 
in all activities 
11.6 10.1 117 11.0 11.2 
to 200 27.7 35.0 29.0 32.0 30.0 
to 500 26.6 23.7 25.1 24.6 25.4 
501 to 700 16.7 14.4 13.1 13.8 18.1 
701 of more 17.4 16.7 21.2 18.6 18.3 
Average length of stay (months) 6.2 68 69 69 6.6* 
Average months receiving services 5.7 6.1 6.1 6.1 5.9 
Percent stil es 
3 75.7 77.0 76 76.7 
6 §2.2 52.9 84.1 §3.5 §2.9 
9 27.2 31.1 31.4 31.3 5 
12 10.9 18.3 19.8 19.1 15.4e*¢ 
18 69 9.3 11.0 10.2 8.7* 
29 62 49 5.6 4.4e* 
19 or later 25 3.9 42 41 3.3 
size 448 287 283 540 968 


SOURCE: MDRC calculations from JOBSTART enroliment form and MIS date. 


NOTES: Calculations for this table used data for all 968 experimentais for whom there were 48 months of 
Se RE SU RIS SIOEE SR SEO OF EG RF CURRIN Otis ORE HSE Aare MORN 90 eee 


Dlewtbutions rey net total 100.0 percers because of 

A Pearson chi-square statistic was used to test the of equal distributions. itr Irgthetleatdy 
Cistributions were those for women who were with their own child(ren) and those for women not living 
with their own ren), those who were childless, at the time of random assignment. An F-statistic was used 
to test the hypothesis of equal means. Statistical significance levels are indicated as *** = 1 percent; ** = § 
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however, for females living with their children compared to other women: Mothers averaged 
somewhat fewer hours of participation, and a higher percentage received fewer than 200 hours of 
services. 

Table 3.8 presents average total hours of participation in JOBSTART for other subgroups. 
Although past research and experience suggest that the characteristics listed in the table might affect 
participation, many of the comparisons do not show significant differences in average hours for the 
subgroups under review. Youths who had been arrested since age 16 (most of whom were males) 
participated for significantly fewer hours than those who had not,2° but other subgroups did not 
show statistically significant differences in hours. 


B. Differences in Participation Among Sites 


Hours of participation in the sites in the demonstration varied considerably, as shown in Table 
3.9. Average total hours ranged from a high of 607 for experimentals at the Los Angeles Job Corps 
to a low of 171 at CREC in Hartford, a spread of 436 hours. Another important aspect of variation 
was the percentage of experimentals still participating in JOBSTART at 12 months after random 
assignment. This proportion varied from a low of zero percent at Chicago Commons and 
SER/Corpus Christi to a high of 41 percent at Allentown. The proportion still active at 18 months 
ranged from zero percent to 16 percent at the Los Angeles Job Corps. Thus, the 48 months of 
follow-up do not represent the same length of post-program follow-up at all sites. 

As noted earlier, this variation could have had several possible sources, such as characteristics 
of the youths, local employment opportunities, and program characteristics.“ With only 13 sites 
in the demonstration, it is very difficult to isolate the effects on participation of the many differences 
among programs. If, for example, the sites with the most support services were also Job Corps sites 
and also operated a youths-only program, it would be impossible to separate out the effects of these 
individual factors on participation hours. Further, the demonstration was not designed to address this 
type of question with the same rigor provided for comparisons of experimentals and controls. 


23The mix of activities did differ by initial reading score. Those testing in the low group averaged slightly 
more hours in education, and had somewhat fewer hours in training, than those in the highest group. 
in the way random assignment interacted with site recruitment efforts, and also in attendance 
reporting, led to variations in participation rates as well. At CET/San Jose, for example, services were not 
available for the first part of the sample for up to one month. Furthermore, as discussed in Chapter 2, 
attendance at a multi-day assessment of occupational training interests was not included in reported hours. 
As a result, 36 percent of the experimentats at that site had no reported hours in program services. At other 
sites, the gap between random assignment and reported program start-up was shorter and participation rates 
were higher. 
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TABLE 3.8 


AVERAGE TOTAL PARTICIPATION HOURS, BY SELECTED CHARACTERISTICS 
OF EXPERIMENTALS AT THE TIME OF RANDOM ASSIGNMENT 


Characteristic 
and Subgroup Average Total Hours Number of Experimentais 
16-19 393.4 724 
20 or 21 417.9 264 
E 
White, non-Hispanic 364.1 82 
Black, non-Hispanic 389.0 440 
Hispanic 387.0 439 
School grade at time of dropout 
Grade 10 or below 407.8 570 
Grade 11 of 12 389.1 418 
Reading grade level 
16 359.1 282 
78 368.3 190 
9 or above 327.9 20 
Gender 
Women 400.6 540 
Men 399.1 448 
Marital status 
Ever married 378.7 97 
Never married 402.2 691 
Paremal status 
Women living with own child(ren) b 373.5 257 
Women not living with own child(ren) 420.0 263 
AFOC benefits received 
None 385.0 607 
Own AFDC case 434.5 200 
Household AFDC case 411.7 181 
Received occupational 
training within past year 
No 403.9 833 
ves 378.3 155 
Reason for leaving regular high school 
School-related 410.4 489 
Job-related 407.4 91 
Other 385.8 406 
Criminal record 
No arrest since age 16 415.4 837 
Arrested since age 16 314,19¢* 151 
Sample size 988 
(continued) 
-82- 


TABLE 3.8 (continued) 


SOURCE: MODRC caiculations from JOBSTART enrollment form and MIS data. 


NOTES: Calculations for this table used data for all 988 experimentals for whom there were 48 
months of follow-up survey data. Sample sizes reported may fall short of this number because of tems 
missing from some sample members’ questionnaires. 

An F-statistic was used to test the hypothesis of equal means. Statistical significance levels are 
indicated as *** = 1 percent; ** = § percent; * = 10 percent. 

b he sample also included 27 experimentals who were members of other ethnic groups. 
includes women who did not have children. 
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TABLE 3.9 
PARTICIPATION RATES, HOURS OF PARTICIPATION, AND LENGTH OF STAY 


FOR EXPERIMENTALS, BY SITE 


Atlanta CET/ Chicago 
13 Stes Job Corps San Jose Commons (Pittsburgh) 
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TABLE 3.9 (continued) 
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63 
56 


5.6 
48 


10.7 


76 
7.0 


6.0 
5.4 


66¢ ee 
5.9** 


Average length of 


stay (months) 


71 60 


116 


154 


155 


TABLE 3.9 (continued) 


SOURCE: MORC calculations from JOBSTART enrolment form and MIS data. 
NOTES: Calculations for this table used date for all 968 experimentals for whom there were 48 months of follow-up survey data, including those with values 


of zero for outcomes end those who were assigned to JOBSTART but did not participate. 
Distributions may not total 100.0 percent because of rounding. 
A Pearson chi-square statistic wes used to test the hypothesis of equal distributions. An F-statistic was used fo test the hypothesis of equal column means. 
Statistical significance levels are indicated as *** = 1 percent; ** = § percent; * = 10 percent. 
Phoenix Job Corps did not report hours spent by participants in life skills or evocational activities. 
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Applicants were randomly assigned to the experimental or control group, but there was no random 
assignment to various types of sites, and within each labor market there was usually only one site. 
This means that the power of the random assignment research design applies to differences between 
experimentals and controls (at a site or in the aggregate) and for differences among subgroups (as 
defined by pre-random assignment characteristics). Other types of comparisons, such as between 
types of sites, are inherently less reliable, and the strength of conclusions depends on the consistency 
of results across sites.> 

Analysis presented in a previous report found that differences in experimentals’ characteristics 
among the sites explained only a small part of the differences in average total hours.2° The analysis 
also found that labor market conditions affected participation: Sites with better employment 
opportunities, other things being equal, had lower average participation hours.2” 

The strongest influence on participation in JOBSTART appeared to be program structure: 
whether a site was concurrent, sequential/in-house, or sequential/orokered. Participation rates by 
component, participation hours, and percentage of time in education or training all differed among 
the three types of sites, as detailed in Table 3.9 and summarized in Table 3.10.28 Four conclusions 
about program structure can be drawn: 

¢ Average hours of participation varied by type of site. Experimentals in 

sequential/in-house sites had the highest average participation hours, while those 


in sequential/orokered sites had by far the lowest because of very low average 
hours in training. 


¢ The mix of education, training, and other activities varied by type of site. The 
concurrent sites, other than two Job Corps sites, did not offer the optional “other 
activities” and emphasized occupational training; as a result, average ese seh 
for experimentals amounted to 71 percent of average total hours. 
sequential/brokered sites emphasized education and other nontraining services, pe 
of which were provided in-house. They had the highest average hours in 
education; training hours were only about 22 percent of average total hours. The 


See Chapter 5 for a more detailed discussion of the difficulty of making cross-site comparisons. 
Auspos et al., 1989. 
2"This could have been because those participating in JOBSTART found a job more easily and left the 
program after fewer hours. Alternatively, sites in labor markets with low unemployment may recruit youths 
who have more unmeasured barriers to employment, are harder to work with in a program, and end up with 
fewer, hours of participation. 

As discussed earlier, eight sites provided concurrent basic education and occupational skills training 
(‘concurrent” sites); two provided education followed by training (“sequential/in-house" sites); and three 
provided education and then referred participants to other agencies for training (“sequential/brokered” sites). 

percentage of average total hours is calculated by dividing average training hours by average total 
hours. Percentages that follow are similarly calculated. 
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TABLE 3.10 
PARTICIPATION RATES, HOURS OF PARTICIPATION, AND LENGTH OF STAY, 


BY PROGRAM STRUCTURE 
Sequential/ Sequential/ All 
Activity Measure Concurrent In-House Brokered Catagories 
Percent participating in 
Education 84.2 87.6 87.4 85.5 
Training 85.1 50.2 24.6 66.5°** 
Education and training 82.9 50.2 246 65.2°** 
Other activities 13.9 89.0 67.2 39.7¢¢¢ 
Average hours in 
Education 101.0 146.1 176.4 124,5¢¢* 
Training 276.1 276.4 71.1 238.2*°* 
Education and training 377.2 422.5 247.6 362.8°°* 
Other activities 9.8 95.0 59.2 36.9°** 
All activities 387.0 518.5 306.8 399.9°** 
Percemage distribution of hours 
in education and training 
None 11.6 12.4 12.6 11.9¢e¢ 
Up to 200 29.2 39.7 48.6 35.0 
201 to S00 24.0 15.8 20.2 216 
501 to 700 19.6 11.0 9.3 15.9 
701 of more 15.6 21.1 9.3 15.6 
Total f 
Percemage distribution of hours 
in all activities 
None 11.1 11.0 12.0 11.20e¢ 
Up to 200 28.5 24.9 40.4 30.0 
201 to 500 24.7 27.8 25.1 25.4 
501 to 700 19.5 86 8.2 18.1 
701 of more 16.3 27.8 14.2 18.3 
Total 
Average length of stay (months) 6.1 7.0 7.7 6.6°* 
Average months receiving services 5.6 66 62 §.9*¢ 
Percent still participating in month 
77.0 78.5 73.8 76.7 
6 54.2 46.9 55.7 52.9 
9 25.7 32.5 38.3 29,.5°e¢ 
12 11.2 16.7 27.3 15.4e¢¢ 
15 44 12.0 19.1 6.7**° 
18 2.0 9.1 66 4.aqeee 
19 or later 1.0 7.7 6.0 3.gee* 
Sample size 569 209 163 968 


SOURCE: MORC calculations from JOBSTART enrolment form and MIS data. 


NOTES: Calculations for this table used data for all 968 experimentais for whom there were 48 months of follow- 
Up Survey data, including those with values of zero for outcomes and those who were assigned to JOBSTART but did 
not participate. 


Distributions may not total 100.0 percent because of rounding. 
A Pearson chi-square statistic was used to test the hypothesis of equal distributions. An F-statistic was used to test 


sequential/in-house sites had the highest total average hours: hours in education 
and other nontraining activities approximating those of the sequential/orokered 
sites, and hours in training like those of the concurrent sites. 


¢ Sequential/brokered sites had difficulty moving participants from education to 
training. Only 25 percent of participants at sequential/orokered sites made the 
transition to occupational training, although those who made the transition did 
receive substantial training. This low rate of participation in training occurred 
owing to the difficulty of linking participants with other organizations, in part 
owing to the nature of typical JTPA Title IIA contracts. Possibly, it also arose 
because participants in these sites (which were primarily basic education 
organizations) were more interested in receiving a GED than occupational training. 


¢ These relationships do nut appear to have been the result of measured differences 
in participant characteristics or local employment opportunities. Even after 
adjustments for measured differences in participant characteristics and local 
employment opportunities, these patterns of participation among sites with 
different program structures still appear.°° 
While these three site categories do clarify patterns of participation, the sites within each 
category were clearly not identical. Among the concurrent sites, EGOS in Denver stood out with 
especially low hours — possibly because of its very large size, which could have left the JOBSTART 
youths feeling isolated and disconnected from the program. CREC in Hartford, among the 
sequential/orokered sites, had very low hours because it scheduled only three hours of education per 
day and very few experimentals participated in training. Furthermore, CREC offered limited support 
services and moved several times during the demonstration, which disrupted program operations. The 
high total hours for sequential/in-house sites were primarily owing to the Los Angeles Job Corps, 
which had the highest average hours among all sites. E! Centro in Dallas, the other site in this 
category, ranked only slightly above the average for all sites in total hours. 


IV. ASummary of the JOBSTART Experience 


The support for the three themes highlighted at the beginning of this chapter can now be 
summarized using the findings in this chapter. First, the diversity of the sites within the general 
framework of the JOBSTART guidelines has been a theme of this chapter. A second, related theme, 
the ability of JOBSTART to be implemented in a variety of settings, has also been discussed. Table 


3°When dummy variables for type of site were added as independent variables to a regression equation with 
individual demographic characteristics and a measure of local employment opportunities, the relationships still 
held. 
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TABLE 3.11 


SUMMARY OF JOBSTART IMPLEMENTATION, BY SITE 
ee 


Average JOBSTART 
Average Length = Average = _ Average Hours in Levelof, «= _Ratingof (Overall Operating 
Total of Stay Hours intial Job Support _ Rating of Costs per 
Ste Hours (Monthe) per Month Education Training Screening Placement Services implementation Experimental ($)® 
Concurrent 987 61 63 101 276 sat + om a wi 
Atlanta Job Corps 334 58 59 102 177 High Lu High Low 4,100° 
CET/Sen Jose” 335 at 62 260 309 Low High Medium High 2,000 
Chicago Commons® 422 4s 98 99 363 High High Medium High 6.400 
(Pittsburgh) 838 95 87 99 439 += Medium = Medium High High 5,200 
Eest LA Skite Center 370 8.7 65 76 294 Medum = Low Medum —- Medium 4,900 
EGOS (Derwer) 270 71 38 128 142 Low Low Medium Low 2,000° 
Phoenix Job Corps 420 63 67 161 202 High High High High 4,700° 
SER/Corpus Chieti 400 6.0 60 118 262 Medium Low High Medium 2,100 
816 7.0 78 146 276 Pr a ia = on 
El Centro (Daitas) 408 60 68 147 179 Medium Medium Medium High 5,100 
607 7.8 79 146 355 High —- Medium High High 5,700° 
[brokered 907 7.7 39 176 71 “ as in 
Allentown (Buttaio) 427 10.7 38 238 105 Medium = Low High Medium 5,900 
(NYC) 262 r 49 149 63 Low Medium Low 7,500 
CREC (Hartiord) 171 6.3 29 124 35 Low Low Medium Low 5,200 
All sites 396 66 60 128 238 ~ o- one ase - 


SOURCES: MODRC calculations from JOBSTART enrolment form and MIS deta (participation figures); MORC operations staff (implementation ratings); Appendix C (costs). 


NOTES: Calculations for thie table used data for all 968 experimentale for whom there were 48 months of follow-up survey data, including those with values of zero for 
outcomes and those who were assigned to JOBSTART but did not 

®These cost estimates are rounded to the nearest $100. Al costs are in 1986 dollars. 

Orne estimates do not Include the cost of providing on-eke medical end dental services. The vaive of these services per JOBSTART experimental was approximately $400 in 
the Atlanta ste, $400 in Phoenix, $600 in Los Angeles, and $24 in Denver. 

Cin thie ste, education end training were more integrated than in other shes, and staff strongly emphasized training over passing the GED examination. 

Sin this site, some education hours ere included in the training component hours. 
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3.11 illustrates both themes by listing the characteristics of the sites, grouped by program structure 


(concurrent, sequential/in-house, sequential/rokered). It describes participation in JOBSTART for 
experimentals by presenting average total hours; average length of stay in the program; average hours 
per month in the program; and average hours of education and training.*! Table 3.11 also includes 
more subjective ratings of implementation, including a rating of the level of initial screening done by 
each site at intake, of sites’ success in implementing job placement and support services (drawn from 
Table 3.4), and of overall JOBSTART implementation. Finally, the average cost of JOBSTART 
services per experimental in each site’s research sample, listed in the right column of this table, adds 
to the description of prograrn implementation already presented in this chapter.?2 

The final major theme of the chapter — the diversity of individual experience in JOBSTART 
~ also emerges from the findings presented. Behind the measures of average level of participation 
there is a wide distribution in the intensity of participation. Subsequent chapters examine whether 
there are similar differences in impacts. 


31Note that each of these measures includes the 11 percent of experimentals who did not participate in 
the program. All of these items have ben included in previous tables except for average hours per month, 
which is a measure of the extent to which program services were concentrated or spread out over time. 

C for the details of these cost data, which include items not normally part of program 
budgets, such as the value of donated goods and services. These cost figures are not net cost figures such as 
those produced as part of the benefit-cost analysis reported in Chapter 7, which compares the impact of the 
program to the incremental cost of the services received by experimentals above the level of services received 
by controls. As discussed in Chapter 4, the controls received substantial non-JOBSTART services; thus, the 
impact figures estimate the effect of the services received by experimentals above this base of services. Costs 
used to calculate a benefit-cost ratio would measure the resource cost of the extra services received by 
experimentals. The cost figures reported in Table 3.11 are gross program costs from which the costs of 
services received by controls are not subtracted. 
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CHAPTER 4 


PARTICIPATION IN EDUCATION AND TRAINING 
AND PROGRAM _IMPACTS ON EDUCATIONAL ATTAINMENT 


The goals of the JOBSTART program included increasing participation in education and 
training activities by a group who otherwise would be little served, and thereby enhancing their 
educational attainment, employability, and long-term earnings. Chapter 3 described the experimental 
group's participation in JOBSTART activities. This chapter takes the story a step farther by 
comparing the participation of these experimentals in JOBSTART (and other programs) to the 
activities of the control group, to determine whether the offer of JOBSTART services actually led 
to an increase in participation over what would have occurred otherwise.? As will be shown, the 
youths in the experimental group did participate in more education and training than those in the 
control group, but the control group was not unserved. Further, participation in JOBSTART did 
lead to an increase in the rate of receipt of aGED. Chapter 5 examines the extent to which this 
increase in services and educational attainment led to improvements in employment and earnings 
by month 48 after random assignment, while Chapter 6 covers impacts on welfare receipt and other 
outcomes. Chapter 7 uses these findings, in addition to information on program costs, to present 
a benefit-cost analysis of JOBSTART.? 

The analysis in Chapters 4 through 7 rests on the random assignment research design 
(described in earlier chapters) to estimate the differences that JOBSTART made in the lives of 
those young people given access to it. To evaluate these differences (often called program 
impacts"), it is necessary to answer two basic questions. First, on average, what happened to those 
who were offered access to the program — in this context, the “experimental” group? Second, on 
average, what would have happened to them had they not been offered access to it, here 


1This was based on attendance information reported by the JOBSTART program operators for those in 
the experimental group. 

2in addition to using the JOBSTART attendance data from the program MIS, this chapter relies on survey 
data on non-JOBSTART services for both experimentals and controls gathered 12, 24, and 48 months after 
random assignment. The JOBSTART MIS data are more accurate about hours attended than are the survey 
data for other programs, and JOBSTART was a large part of the services received by experimentals. 
Therefore, the measurement of hours of participation for experimentals is more accurate than for controls. 

3Chapters 4 through 7 present findings for the full 48-month impact sample as well as for subgroups 
defined by individual characteristics such as gender. 
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represented by the experience of the “control” group?* The average effect, or impact, of a program 
is the difference between the two groups in the many outcomes of interest. 

This chapter addresses several key evaluation questions: 

¢ Did a positive experimental-control difference in receipt of education and training 


materialize? If so, was this difference maintained over time, or did controls catch 
up with experimentals by the end of four years after random assignment? 


* How did service receipt by experimentals and controls compare among important 
subgroups such as men, custodial mothers, and all other women in the sample? 


¢ Did JOBSTART produce post-program gains in educational attainment as 
measured by receipt of a high school diploma or GED? If so, were controls 
comparable to the experimentals in educational attainment by the fourth year? 


¢ Did the educational attainment of experimentals and controls vary among 

important subgroups? 

To summarize the findings of this chapter, access to JOBSTART did substantially increase the 
experimentals’ participation in education and training activities, raising their rates and average hours 
of participation well above those of controls, almost half of whom were also active in these types 
of activities in other programs. This overall difference occurred primarily because of substantially 
higher participation by experimentals in the first 12 months after random assignment, when many 
were active in JOBSTART, and was present for all important subgroups in the sample. In the 
second year of follow-up — months 13 through 24 — experimentals had a slightly higher 
participation rate than controls. During months 25 through 48, experimentals and controls spent 
about equal time in education and training activities. Experimentals’ greater overall service receipt 
led to a substantially higher GED receipt rate than among controls, for both the full impact sample 
and most subgroups. 


The purpose of the JOBSTART control group was to be a benchmark for measuring program 
impacts, but if most controls received services similar to those received by experimentals, the 
benchmark would be useless, and it would be very difficult to evaluate JOBSTART.’ Because 


‘Since, as shown in Chapter 2, assignment to JOBSTART was random, there were no systematic 
differences between experimentals and controls at enrollment; therefore, outcomes for controls could be used 
to measure what would have happened to experimentals without the program. 

SThe service receipt differences reported here were calculated by comparing the experiences of 
experimental; and controls, as was done to determine the impact of JOBSTART on such outcomes as 

(continued...) 
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JOBSTART targeted disadvantaged dropouts with poor reading skills (a group little served within 
the JTPA system and elsewhere), demonstration planners anticipated that controls would not be 
served to any great extent by other programs. 

However, JOBSTART controls were expected to receive some services. Even though many 
performance-driven programs screen out people with low reading levels, JOBSTART control group 
members were more determined than many school dropouts to pursue services, as evidenced by their 
volunteering for JOBSTART. In fact, at some point within the 48 months following random 
assignment, 56.1 percent of controls received non-JOBSTART remedial or occupational instruction. 
Therefore, the program impacts on educational attainment, employment, earnings, and other 
outcomes presented in this report are the incremental impacts of JOBSTART over the mix of 
services available throughout the community to control group members that they pursued on their 
own initiative. And although controls received substantial services, experimental-control service 
receipt differences were still large, as is shown in this chapter. 


A. In-Program and Post-Program Outcomes 

All the events tracked by the JOBSTART MIS and the follow-up surveys (including program 
participation, GED receipt, employment, and other outcomes) were reckoned from the date of 
random assignment, not the date of termination from the program.® There was a great deal of 
variation in length of stay in JOBSTART among experimentals, but about 88 percent had stopped 
participating in the program by the end of month 12 after random assignment. (See Table 3.5.) 
Thus, in reviewing the findings on program participation and impact estimates, it is useful to think 
of the first 12 months of follow-up as primarily an in-program period and the months thereafter as 
primarily post-program. 

Table 4.1 shows that, over the four years of follow-up, 94 percent of experimentals and 56.1 
percent of controls received some education or training, for an impact of 37.9 percentage points. 


5(...continued) 
educational attainment and employment, analyzed later in this report. These service receipt differences are 
pot normally thought of as program “impacts” because service receipt is the means to reaching the final 
program goals. However, in this chapter, when discussing experimental-control service receipt differences, the 
word impact has been used to simplify the terminology. 

This is a different approach from that used in the Job Corps study (Mallar et al., 1982), in which follow- 
up began at termination from the program, thereby complicating a comparison of JOBSTART and Job Corps 
impact findings. But as will become clear in Chapter 5, starting follow-up at program entry allows fcr a careful 
analysis of the foregone earnings resulting from participation in the program. 


-94- 


166 


TABLE 4.1 


IMPACTS ON RECEIPT OF EDUCATION OR TRAINING 


THROUGH MONTH 48 
Controls 

Outcome and Follow-Up Period (%) (%) Difference p* 
Ever received any education 
or training, months 1-48 94.0 56.1 37.9*** 0.000 
Ever received any education 
or training, months 1-12 90.4 26.2 64.2*** 0.000 
Ever received any education 
or training, months 13-24 31.8 25.8 6.0*** 0.003 
Ever received any education 
or training, months 25-48 30.8 31.4 -0.6 0.777 
Sample size 988 953 


SOURCE: MDRC calculations from JOBSTART enroliment form, MIS, and survey data. 


NOTES: Calculations for this table used data for all sample members for whom there were 
48 months of follow-up survey data, including those with values of zero for outcomes and those who 
were assigned to JOBSTART but did not participate. 

"Any education or training’ includes JOBSTART and non-JOBSTART education, occupational 
sa deh alana 

Random assignment did not always take piace on the first of the month. For some sample 

members, the month of random assignment may be less than a month, beginning with the date of 
random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted means from 
linear analysis of covariance procedures controlling for up to 31 kinds of difference in characteristics 
before random assignment (see Ostle, 1975, p. 461; Cave, 1987; and Appendix 8). There may be 


slight discrepancies in reported sums and differences of these adjusted means because of rounding. 


average experimental 
= 1 percent; ** = 5 percent; * = 10 percent. 
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This impact was statistically significant, that is, too big to be likely to have arisen entirely by chance. 
Its significance is indicated by the stars next to the impact estimate (the "difference") and the p- 
value, which is virtually equal to zero, signifying that the likelihood that the observed difference 
occurred because of random error is very low. During the first follow-up year, the difference was 
large and significant (64.2 percentage points); during the second it declined as JOBSTART 
participation ended (to 6 percentage points); and in the latter two years an almost equal number 
of experimentals and controls received education or training. On a monthly basis (not shown in the 
table), the proportion of experimentals in programs, mainly JOBSTART, was highest during the first 
three months after random assignment and fell rather steadily over time, to 11.2 percent during 
month 24. The proportion of controls in programs was much smaller at the beginning of the follow- 
up period, peaked at about 15 percent during months 10 through 13, and soon thereafter reached 
a plateau at about 10 to 12 percent, about even with the level to which experimentals had fallen. 

Hours of education or training followed a similar pattern. Table 4.2 shows that over the 
four-year period experimentals participated an average of 800 hours and controls an average of 432 
hours, for an impact of 367 hours.’ The difference was largest in months 1 through 12 (300 hours), 
when most experimentals were active in JOBSTART; dropped to 65 hours in months 13 through 
24; and was virtually zero over the final two years of follow-up. On a monthly basis (not shown in 
the table), average experimental hours peaked at about 70 hours per month during month two and 
then fell steadily, while control hours, always much lower, peaked during month nine and then 
stabilized at about 10 hours per month for the remainder of the first two years of follow-up before 
dropping to very low levels in the final two years. 

Among the 94 percent of experimentals and 56.1 percent of controls who received any services 
during the four-year period — that is, excluding those who received no services — experimentals 
averaged 882 hours and controls 800 hours, a difference of 82 hours.® Because so many fewer 
controls received services, and those who did were probably quite motivated, this difference in hours 
for those who received services may understate the intensity of participation among served 
experimentals. The served experimentals who were directly comparable in motivation to the served 
controls might well have been those who received many more than the average hours for all served 


experimentals. 


7It is important to remember that the average total hours for experimentals and controls shown in Table 
4.2 include data for sample members who did not participate and therefore had zero hours. Other information 
included below presents average total hours only for those active in some program. 

Sas mentioned above, this finding is not shown in Table 4.2. 
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TABLE 4.2 


IMPACTS ON TOTAL HOURS OF EDUCATION OR TRAINING RECEIVED 


THROUGH MONTH 48 
Outcome and Follow-Up Period Experimentals Controls _ Difference p* 
Total hours of education or 
training received, months 1-48 799.59 432.44 367.15*** 0.000 
Total hours of education or 
training received, months 1-12 415.66 115.47 300.19°** 0.000 
Total hours of education or 
training received, months 13-24 155.37 90.24 65.13*** 0.000 
Total hours of education or 
training received, months 25-48 228.57 226.73 1.84 0.955 
Sample size 988 953 


SOURCE: MDRC calculations from JOBSTART enroliment form, MIS, and survey data. 
NOTES: Calculations for this table used data for all sample members for whom there were 


For experimentais, “hours of education or training’ include JOBSTART hours from MIS data and 
non-JOBSTART hours from survey data. 
Random assignment did not always take place on the first of the month. For some sample 


average 
= 1 percent; ** = 5 percent; * = 10 percent. 
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In summary, a service receipt differential between experimentals and controls did materialize. 
Not only did experimentals receive education and training at vastly higher rates than controls 
throughout the first half of the 48-month follow-up period, but they also on average received many 
more hours. However, as noted earlier, the control group did receive some similar services from 
other programs in the community. Thus, the findings presented in Chapters 4, 5, and 6 represent 
the incremental impacts of JOBSTART above the existing level of service. 


Table 4.3 splits the 48-month sample into three groups: men, women living with children of 
their own at random assignment (custodial mothers), and all other women in the sample; the impacts 
on service receipt are presented for each group using the same methods as in Tables 4.1 and 4.2. 
In general, approximately equal percentages of experimentals in all three groups were active in all 
types of education and training activities in the first year, when most activity occurred.? However, 
there were clear differences in participation among the control groups for the three subgroups: 
Men in the control group participated in education and training less than custodial mothers, who 
in turn participated less than all other women. 

The larger differences in participation rates and average hours between experimentals and 
controls for men, therefore, result largely from the lower level of activity of the control group. 
During the four-year follow-up period, 94.3 percent of men in the experimental group and 51.3 
percent in the control group were active in education or training, for a difference of 43 percentage 
points. This resulted in a difference of 392 average total hours between men in the experimental 
and control groups. For women living with their own children, more experimentals were active than 
controls (95 percent versus 59.7 percent) during the four-year follow-up period, and experimentals 
averaged 331 more hours than controls. For other women, the comparable figures are 93.2 percent 
versus 60.9 percent, and 337 more hours for experimentals. For all three of these groups, in the 
third and fourth years of follow-up, participation in education or training by both experimentals and 
controls declined sharply and the difference in their participation was no longer statistically 
significant. 


This finding is based on an analysis of the proportion of experimentals in each group who ever 
participated in an activity during the period in question. The finding holds during the first year of follow-up 
and over the entire 48-month follow-up period as well. In the second year of follow-up (and over the entire 
two-year follow-up period), more “other women® (women who were not custodial mothers) participated in an 
activity (registering the highest average total hours in the second year). Custodial mothers had the next highest 
average total hours and men had the lowest. These differences were largely attributable to differences in 
participation in non-JOBSTART activities. As reported in Chapter 3, participation in JOBSTART activities 
was similar among the three groups. 
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IMPACTS ON RECEIPT OF EDUCATION OR TRAINING THROUGH MONTH 48, 
BY GENDER AND PARENTAL STATUS 


TABLE 4.3 


Subgroup, Outcome, 
and Follow-Up Period Experimentais Controls Difference p* 
Men 
Ever received any education 
or training (%) 
Months 1-48 94.3 §1.3 43.0°** 0.000 
Months 1-12 89.7 21.9 67.7*** 0.000 
Months 13-24 27.4 18.6 8.9*** 0.002 
Months 25-48 26.1 30.1 4.1 0.180 
Total hours of education or 
training received 
Months 1-48 758.25 366.56 391.69*** 0.000 
Months 1-12 423.64 84.05 339,59*** 0.000 
Months 13-24 141.79 60.51 81.28°** 0.000 
Months 25-48 192.83 222.0 -29.17 0.547 
Sample size 448 452 
Women living with own child(ren) 
Ever received any education 
or training (%) 
Months 1-48 95.0 59.7 35.3°** 0.000 
Months 1-12 91.4 26.0 65.4*** 0.000 
Months 13-24 35.2 30.5 4.7 0.271 
Months 25-48 39.1 34.1 5.0 0.254 
Total hours of aducation or 
training received 
Months 1-48 776.68 445.22 331.46*** 0.000 
Months 1-12 388.76 113.33 275.43*** 0.000 
Months 13-24 161.45 94.40 67.04** 0.028 
Months 25-48 226.47 237.49 -11.02 0.856 
Sample size 257 251 
(continued) 
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TABLE 4.3 (continued) 


and Follow-Up Period Experimentals Controls _—Difference p* 


who did not have any 

Ever received any education 

or training (9%) 
Months 1-48 93.2 60.9 32.3*** 0.000 
Months 1-12 91.0 33.8 57.2°** 0.000 
Months 13-24 37.1 32.4 47 0.247 
Months 25-48 31.2 30.7 0.5 0.898 

Total hours of education or 

training received 
Months 1-48 881.29 543.90 337.39°** 0.000 
Months 1-12 426.45 175.60 250.86*** 0.000 
Months 13-24 179.45 130.64 48.81 0.135 
Months 25-48 275.39 237.66 37.73 0.578 

Sample size 283 250 


SOURCE: MDRC calculations from JOBSTART enroliment form, MIS, and survey data. 


NOTES: Calculations for this table used data for all sample members for whom there were 48 
months of follow-up survey data, including those with values of zero for outcomes and those who were 
assigned to JOBSTART but did not participate. 

*Any education or training’ includes JOBSTART and non-JOBSTART education, occupational 
skilis training, and related activities. For experimentals, “hours of education or training’ include 
JOBSTART hours from MIS data and non-JOBSTART hours from survey data. 

Rancom assignment did not always take place on the first of the month. For some sample 
members, the month of random assignment may be less than a month, beginning with the date of 
random assignmemt and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted means from spiit- 
file linear analysis of covariance procedures controlling for up to 29 kinds of difference in 


There may be slight discrepancies in reported sums and differences of these adjusted means 
because of 


*The column labeled ‘p’ is the statistical significance level of the difference between 
experimental and control group outcomes: that is, p is the probability that average outcomes are 
Gifferent only because of random error. A two-tailed t-test was applied to each difference between 

and control group outcomes. Statistical significance levels are indicated as *** 


average experimental 
= 1 percent; ** = 5 percent; * = 10 percent. 


C. Differences in Receipt of Education and Training by Other Subgroups 


The primary goal of the JOBSTART evaluation is to estimate the difference that access to the 
program made for its target population (in other words, the program's impacts). While the size of 
the full 48-month sample is large enough to estimate overall impacts on policy-relevant outcomes, 
it provides considerably less statistical power for estimating subgroup impacts and differences in 
impacts among subgroups. Keeping this limitation in mind, this section presents an analysis of the 
difference in service receipt by experimentals and controls for various subgroups and compares the 
size of this difference among the selected subgroups. 

To summarize the findings in this section: The service receipt differences between 
experimentals and controls observed for the full sample were present and large for virtually all 
important subgroups. The observed differences in service receipt impacts among subgroups primarily 
reflect variation in the level of service receipt of controls. 

The impacts presented in Table 4.3 represent split-sample analysis, discussed in Chapter 2, 
which does not control for differences in baseline characteristics other than the characteristic used 
to define the subgroup. Table 4.4 uses the second method of analysis described in Chapter 2 to 
calculate within-subgroup impacts and between-subgroup impact differences for the most important 
measure of program activity: receipt of education or training during the four-year follow-up period. 
This method controls for differences in baseline characteristics other than that used to define the 
subgroup. !° 

For example, the first three rows of Table 4.4 present impacts for women and men using 
statistical techniques to control for gender differences in factors such as employment experience, 
educational level, ethnicity, and parental status. It thus shows a comparison of the impacts by 
gender with other characteristics held constant. The first row of Table 4.4, in the column labeled 
“subgroup impact difference,” shows that the impact for women was 7.6 percentage points below the 
impact for men, and this difference in service reccipt impacts was statistically significant. The 
“subgroup impact" column shows that this difference in impacts was calculated as 36.2 percentage 


1°The impact estimates in Table 4.4 were calculated by conducting a two-way analysis of covariance, 
controlling for differences in pre-random assignment characteristics other than the characteristic used to define 
the subgroup (see Ostle, 1975, p. 461). The adjustments were done using a linear regression model. 
Characteristics that affect outcomes and impacts with a nonlinear relationship are not controlled for with this 
procedure, and there is no procedure that can control for unmeasured characteristics that affect outcomes and 
impacts. The adjusted outcomes for men, custodial mothers, and other women presented in Table 4.4 are very 
similar to the unadjusted outcomes in Table 4.3, but the adjustments do make more of a difference for many 
of the other subgroups listed in the table. 
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S 
Characteristic ana Subgroup Size 
Gender 
Women 1,041 
Men 900 
Ethnicity 
White, non-Hispanic 172 
Black, non-Hispanic 860 
Hispanic 847 
Other 62 
1 
o Ethnicity, by gender 
fy Women 
White, non-Hispanic 97 
Black, non-Hispanic 467 
Hispanic 451 
Other 26 
Men 
White, non-Hispanic 75 
Black, non-Hispanic 393 
Hispanic 396 
Other 36 
Parental status 
Women living with own 
child(ren) 
No 533 
Yes 508 
Men who have own child(ren) 
174 No 785 
Yes 115 


TABLE 4.4 


IMPACTS ON RECEIPT OF EDUCATION OR TRAINING 
THROUGH MONTH 48, BY SELECTED CHARACTERISTICS 


AT THE TIME OF RANDOM ASSIGNMENT 


or Training, Months 1-48 
Experimentais § Controls Subgroup 


Ever Received Any Education 

(%) (%) 
89.1 60.5 
87.3 §1.1 
88.8 51.0 
88.3 58.2 
87.6 55.1 
93.7 54.9 
85.1 50.7 
89.6 62.6 
89.3 60.5 
94.8 61.4 
96.6 50.9 
87.3 53.4 
85.6 48.8 
92.1 49.6 
89.9 62.0 
87.3 58.0 
88.2 52.3 
85.5 47.0 


impact 


28.6*** 
36.2*** 


37.8*** 
30.1*** 
32.5*** 
38.8*** 


34.4%" 
27,.0%** 
28.8*** 
33.4** 


45.7*** 
33.9*** 
36.8*** 
42.5*** 


27.g*** 
29,.3¢** 


35.9*** 
38,5*** 


TABLE 4.4 (continued) 


Ever Received Any Education 
or Training, Months 1-48 
Sample Experimentails Controls Subgroup 
Characteristic and Subgroup Size (%) (%) Impact 
Employed within past year 
No 914 88.3 §5.0 33.3*** 
Yes 1,027 88.2 57.1 31,.1*** 
Prior employment, by gender 
Women employed within 
past year 
No 583 92.0 59.7 32.3*** 
Yes 458 85.5 61.5 24.0*** 
Men employed within past year 
No 331 83.2 48.1 38.1*** 
Yes 569 89.9 52.7 37.2*** 
(] 
So Left school in grade 11 or 12 
* No 1,140 88.1 56.3 31.8*** 
Yes 801 88.3 55.7 32.6*** 
Received occupational training 
within past year 
No 1,615 88.7 56.0 32.7*** 
Yes 326 85.7 56.7 29.0*** 
Age 
16-19 1,425 88.8 56.0 32.8*** 
20 or 21 516 86.6 56.4 30.2*** 
Age, by gender 
Women 
16-19 763 89.7 60.6 29.1*** 
20 or 21 278 87.0 60.0 27.0*** 
Men 
16-19 662 87.9 50.8 37.1**8 
20 or 21 238 86.0 52.2 33.8*** 
176 


Characteristic and Subgroup 


Marital status 1.7 0.793 
Ever married 184 89.8 56.1 33.7*** 0.000 - “ 
Never married 1,757 88.1 56.1 32.0*** 0.000 - _ 

Living in own household or 

with boy/girlfriend 4.9 0.312 
No 1,575 89.0 56.0 33.0*** 0.000 - - 
Yes 366 84.7 56.6 28.1*** 0.000 - od 

Own AFDC case or receiving 

General Assistance 7.6* 0.074 
No 1,418 87.2 §3.0 34.2*** 0.000 “ - 
Yes 523 90.8 64.2 26.6*** 0.000 o- - 


' 

> OwnAFO0C case 6.9 0.132 

i. No 1,522 87.7 54.1 33.6*** 0.000 “ me 
Yes 419 89.8 63.1 26.7*** 0.000 “- “ 


Receiving Food Stamps 5.5 0.166 
No 1,214 90.2 56.0 34.2*** 0.000 - 
Yes 727 85.0 56.3 28.7*** 0.000 “- 


Arrested since age 16 1.6 0.761 
No 1,649 88.2 55.9 32.3*** 0.000 - me 
Yes 292 88.0 57.3 30.7*#* 0.000 = oa 


Arrested since age 16, 
by gender on 0.167 
Women 


No 986 89.1 60.4 28.7*** 0.000 o- Lae 

Yes §5 88.3 61.1 22.2** 0.017 oe ass 
175 Men 

No 663 88.0 50.2 37.8*** 0.600 a 


31.6*** =~. 75 


Yes 
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TABLE 4.4 (continued) 


Ever Received Any Education 
or Training, Months 1-48 
Sample = Experimentals = Contro!s Subgroup Subgroup Impact 
Characteristic and Subgroup Size (%) (%) Impact p* Difference p* 
Lived with both parents at 
age 14 4,7 0.232 
No 1,264 86.6 56.1 30.5*** 0.000 - --- 
Yes 677 91.4 56.2 35.2*** 0.000 ad oe 
Reason for leaving regular 
high school - 0.286 
School-related 925 86.9 51.6 35.3*** 0.000 o a 
Job-related 197 85.3 58.1 27.2*** 0.000 -- = 
Other 819 90.5 60.4 30.1*** 0.000 o- -- 


SOURCE: MDRC calculations from JOBSTART enrolment form, MIS, and survey data. 


NOTES: Calculations for this table used data for all sample members for whom there were 48 months of follow-up survey data, 
including those with values of zero for outcomes and those who were assigned to JOBSTART but did not participate. 

Average experimental and contro! group outcomes reported here are adjusted means from two-way analysis of covariance 
procedures controlling for up to 31 kinds of difference in characteristics, other than the characteristic used to define subgroups, before 
random assignment. The two categories used as factors were research assignment and, one at a time, the baseline characteristics indicated 
(see Ostle, 1975, p. 454). There may be slight discrepancies in reported sums and differences of these adjusted means because of rounding. 

*A two-tailed t-test was applied to each within-subgroup impact and also to each difference between subgroup impacts. For each 
characteristic with more than two subgroups, an F-test was applied to the interaction between that characteristic and experimental or control 
status. The columns labeled ‘p* are the statistical significance levels of each impact and each difference in impacts or F-statistic: that is, p is 
the probability that sample estimates are non-zero only because of random error. Statistical significance levels are indicated as *** = 1 
percent; ** = 5 percent; * = 10 percent. 

For each characteristic that has only two subgroups, the subgroup impact difference is the impact within the first subgroup, less the 
impact within the second subgroup. 
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points (the impact for men, shown in the third row) minus 28.6 percentage points (the impact for 
women, shown in the second row). For characteristics with more than two subgroups, such as 
ethnicity, the subgroup impact column shows the experimental-control service receipt difference for 
each subgroup (for example, 37.8 percentage points for whites), but no subgroup impact difference 
is presented; when there are more than two subgroups, no single impact difference among them can 
be calculated. 

In addition to gender, important characteristics for subgroup comparisons of impacts on service 
receipt include: 


¢ Age. In the general youth population, different patterns of labor market behavior 
are exhibited at each age. Labor force participation, employment, and earnings 
often increase from age 16 to the early twenties. Thus, holding everything else 
constant, youths over age 20 are likely to pay higher opportunity costs for program 
attendance than younger participants. These likely differences in labor market 
experiences did not affect the impacts on service receipt, with 16- to 19-year-old 
sample members having virtually the same impact on service receipt (32.8 
percentage points) as 20- to 21-year-old youths (30.2 percentage points). 


¢ Prior employment. For those with a more extensive work history, as evidenced by 
employment in the year before random assignment, the opportunity costs of 
participating in an employment and training program may be greater. 
Experimentals with and without prior-year employment had high rates oof 
participation in education and training and similar impacts on service receipt.!! 


* Highest grade attended. While all JOBSTART enrollees were high school 
dropouts, some left school before completing the tenth grade, while others 
dropped out during their junior or senior year. Despite the differences in past 
success in school, the levels of participation and impacts on service receipt were 
nearly identical for the two subgroups. Apparently JOBSTART sites found ways 
to engage the youths with lower educational attainment in the program. 


¢ Welfare receipt.'2, Those who receive AFDC, General Assistance, or Home 
Relief may tend to get higher levels of support services such as child care, and 
sometimes may be mandated to participate in some type of program to maintain 
eligibility for their cash benefits. Impacts for those not receiving welfare at random 
assignment were higher than for welfare recipients because control group welfare 
recipients were more likely to participate in an education or training program. 


¢ Reason for leaving school. Among the subgroups in this category, those youths 
who dropped out of school for reasons related to their educational experience (as 


MEor men, the difference in service receipt was somewhat greater for those without prior-year 

employment, while for women, those with and without recent employment had similar service receipt impacts. 

12Since only about 13 percent of the men in the sample received AFDC or General Assistance a* random 
assignment, the subgroup receiving welfare was primarily made up of women. 
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opposed to a desire to get a job or the birth of a child) had the largest service 
receipt impact. 


D. Receipt of Education and Training Separately 


Surveys at 12 and 24 months after random assignment collected information on monthly 


participation in education and training separately. These data (not shown in a table) showed a large 
difference between experimentals and controls in participation in education throughout the first half 
of the four-year foilow-up period. During the first year, 88 percent of experimentals participated 
in education compared to 15 percent of controls, for a difference of 73 percentage points. In the 
second year, the participation of experimentals dropped sharply as they left the JOBSTART 
program, while that of controls remained approximately what it was during the first year, causing the 
service receipt difference to decline to only 6 percentage points. Analysis of subgroup impacts again 
indicates a somewhat stronger service receipt differential for men than for women, because female 
controls tended to get more education services on their own than did male controls. 

JOBSTART had a smaller impact on the receipt of training; it achieved less of an 
experimental-control differential for training alone than for training and education together.!? 
The smaller impact for training than for education resulted from the failure of some JOBSTART 
sites, notably the sequential/brokered sites, to move participants from the education to the training 
component (see Chapter 3). Similar patterns of impacts on receipt of training appeared for men 


and women, but again the impacts tended to be larger for men.!4 


II. Impacts on Educational Attainment 


As indicated in the 1989 interim report, Implementing JOBSTART, the impacts of JOBSTART 
on educational attainment during the in-program period (that is, the first 12 months of follow-up) 
were quite similar to those of the program that inspired it, the residential Job Corps. An evaluation 
of the Job Corps found that 24 percent of Corpsmembers, but only 5 percent of the comparison 
group, had high school diplomas or GEDs six months after termination from the program (the 
period roughly equivalent to a year of post-random assignment follow-up).!5 At a similar point 
in the JOBSTART follow-up, 27.5 percent of the experimental group and 9.9 percent of the control 


13The two-year impact for the full sample on receipt of training alone was 41.6 percentage points, as 
compared to 53 percentage points for education or training. 
“The impact on training receipt was 36 percentage points for women and 49 percentage points for men. 
1SMallar et al., 1982. See also Betsey et al., 1985, p. 112. 
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group had a GED or high school diploma, for an impact of 17.6 percentage points. This report 


carries the JOBSTART story forward to the end of the four-year follow-up period. 

JOBSTART impacts on educational attainment for the full 48-month sample are presented in 
Table 4.5. Table 4.6 presents separate split-sample results for men, women living with their own 
children at random assignment, and all other women. The severe and intractable problems in 
reading and mathematics for the young adults in JOBSTART are reflected in the low rates of 
completing high school or passing the GED examination for both experimentals and controls. By 
month 48, 28.6 percent of all controls had received a high school diploma or passed the GED 
examination (see Table 4.5), with other women having the highest percentage among the three 
subgroups in Table 4.6. Among all experimentals, 42 percent had attained a high school diploma 
or GED, with about equal proportions among the three subgroups. 

The full-sample impact on attainment of a GED or high school diploma by month 48 was 13.4 
percentage points. Most of the increased educational attainment for both the experimental and 
control groups came in the first 24 months of follow-up. At that point, the rate of GED or high 
school diploma receipt was 36 percent for experimentals and 21 percent for controls, for an impact 
of 15 percentage points (not shown in the table). 

Among the three key subgroups shown in Table 4.6, the impact over the 48-month follow-up 
period on attainment of a GED or high school diploma was 13.7 percentage points for men, 15.3 
percentage points for custodial mothers, and 10.4 percentage points for all other women. 

Compared to the impacts on attainment of either a high school diploma or GED, the impacts 
on attainment of GEDs only were slightly larger. This was the case because controls (who, as a rule, 
were not participating in an alternative education program similar to JOBSTART) were slightly 
more likely to return to regular high school than experimentals, although it was rare for either 
controls or experimentals. The 48-month impact on GED receipt was 16.5 percentage points for 
the full sample, 15.9 percentage points for men, 15.7 percentage points for custodial mothers, and 
16.8 percentage points for other women.!® 
These large educational attainment impacts were present for many different subgroups in the 


16The JOBSTART surveys also asked about receipt of a trade certificate or license during the follow-up 
period. Because there were not consistent standards, the precise significance of these certificates and licenses 
is not clear. Some could have been awarded for completion of a program, rather than for achievement of a 
generally recognized occupational competency. Nevertheless, 33.1 percent of experimentals received such 
certificates and licenses, compared to 17.3 percent of controls, for an impact of 15.8 percentage points. The 
impact was slightly larger than this for men, and smallest for custodial mothers. 
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TABLE 4.5 


IMPACTS ON EDUCATIONAL ATTAINMENT 
THROUGH MONTH 48 


Experimentais Controls 
Outcome and Follow-Up Period (9%) (%) Difference p* 


Received GED by end of month 48 37.6 21.1 16.5*** 0.000 
Received GED or high school diploma 

by end of month 48 42.0 28.6 13.4*** 0.000 
Received trade certificate or 

license by end of month 48 33.1 17.3 15.8°** 0.000 
Received associate's or 2-year college 

degree by end of month 48 0.5 0.3 0.2 0.484 
Received bachelor’s or 4-year 

college degree by end of month 48 0.0 0.1 0.1 0.291 
Sample size 988 953 


SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 
NOTES: Calculations for this table used data for all sample members for whom there were 


Random assignment did not always take place on the first of the month. For some sample 
members, the month of random assignment may be less than a month, beginning with the date of 
random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted means from 
linear analysis of covariance procedures controlling for up to 31 kinds of difference in characteristics 


slight discrepancies in reported sums and differences of these adjusted means because of rounding. 
*The column labeled ‘p* is the statistical significance level of the difference between 


average experimental and control group outcomes. Statistical significance levels are indicated as *** 
= 1 percent; ** = 5 percent; * = 10 percent. 
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TABLE 4.6 


IMPACTS ON EDUCATIONAL ATTAINMENT THROUGH MONTH 48, 
BY GENDER AND PARENTAL STATUS 


Subgroup, Outcome, 
and Follow-Up Period Experimentals Controls Difference p* 
Men 
Received GED by end of month 48 (%) 36.8 20.9 15.9*** 0.000 
Received GED or high school 
diploma by end of month 48 (%) 42.0 28.3 13.7°"" 0.000 
Received trade certificate or 
license by end of month 48 (%) 35.9 14.9 21.0°** 0.000 
Received associate's or 2-year 
college degree by end of 
month 48 (%) 0.2 0.2 0.0 0.926 
Received bachelor's or 4-year 
college degree by end of month 48 (%) 0.0 0.2 0.2 0.375 
Sample size 448 452 
Women living with own child(ren) 
Received GED by end of month 48 (%) 39.1 23.4 16.7*** 0.000 
Received GED or high school 
diploma by end of month 48 (%) 42.0 26.7 15,3°** 0.000 
Received trade certificate or 
license by end of month 48 (%) 32.1 21.3 10.8*** 0.006 
Received associate's or 2-year 
college degree by end of 
month 48 (%) 1.3 0.2 1.1 0.177 
Received bachelor's or 4-year 
college degree by end of month 48 (%) 0.0 0.0 0.0 1.000 


Sample size 


(continued) 
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TABLE 4.6 (continued) 


Subgroup, Outcome, 
and Follow-Up Period Experimentals Controls Difference p* 


Women not living with own 
child(ren), including those 


who did not have any 
Received GED by end of month 48 (%) 36.9 20.2 16.8*** 0.000 
Received GED or high schoo! 
diploma by end of month 48 (%) 41.6 31.3 10.4** 0.012 
Received trade certificate or 
license by end of month 48 (%) 30.0 17.2 42.8°** 0.001 
Received associate's or 2-year 
college degree by end of 
month 48 (%) 0.4 0.4 0.0 0.924 
Received bachelor’s or 4-year 
college degree by end of month 48 (%) 0.0 0.0 0.0 1,000 
Sample size 283 250 


SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 


NOTES: Calculations for this table used data for all sample members for whom there were 48 
months of follow-up survey data, including those with values of zero for outcomes and those who were 
assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For some sample 
members, the month of random assignment may be less than a month, beginning with the date of 
random assignment and ending on the last day of the month. 

Average experimental and contro! group outcomes reported here are adjusted means from split- 


There may be slight discrepancies in reported sums and differences of these adjusted means 
because of roundirig. 

*The column labeled °p’ is the statistical significance level of the difference between experimental 
and control group outcomes: that is, p is the probability that average outcomes are different only 
because of random error. A two-tailed t-test was applied to each difference between average 

and control group outcomes. Statistical significance levels are indicated as *** = 1 


experimental 
percent; ** = § percent; * = 10 percent. 


TABLE 4.7 


IMPACTS ON GED ATTAINMENT THROUGH MONTH 48, 
BY SELECTED CHARACTERISTICS AT THE TIME OF RANDOM ASSIGNMENT 


Received GED by 
— End of Month 48_ 
Sample _Experimentais Controls Subgroup Subgroup Impact 
Characteristic and Subgioup Size (%) (%) Impact p* Difference p* 
Gender 0.5 0.900 
Women 1,041 38.4 21.7 16.7*** 0.000 o- “- 
Men 900 36.8 20.6 16.2*** 0.000 - - 
Ethnicity - 0.842 
White, non-Hispanic 172 51.6 32.1 19.5*** 0.003 o- - 
Black, non-Hispanic 860 38.1 21.0 47.1%%* 0.000 o- ov 
Hispanic 847 35.0 19.2 15.8*** 0.000 - o- 
Other 62 27.5 19.3 8.2 0.462 oo - 
, Ethnicity, by gender - 0.996 
= Women 
White, non-Hispanic 97 56.7 38.2 18.5** 0.037 -- -- 
Black, non-Hispanic 467 39.4 20.9 18.5*** 0.000 o-- oo 
Hispanic 451 35.0 20.0 15.0*** 0.000 = o-- 
Other 26 15.0 9.5 5.5 0.749 “= o-- 
Men 
White, non-Hispanic 75 43.1 25.8 17.3* 0.092 - - 
Black, non-Hispanic 393 36.3 20.9 15.4*** 0.000 o-- os 
Hispanic 396 35.2 18.5 16.7*** 0.000 -- “- 
Other 36 39.0 24.7 14.3 0.336 oo oo 
Parental status - 0.928 
Women living with own 
child(ren) 
° No §33 38.3 21.1 Ti2"* 0.000 on — 
3 188 Yes 508 39.7 23.5 16.2*** 0.000 ae = 
Men who have own child(ren) 
No 785 36.1 20.6 18.5*** 0.000 o- a 
Yes 115 35.3 14.2 21.1*** 0.010 o- oe 


TABLE 4.7 (continued) 


Received GED by 
Sample Experimentais Controls Subgroup Subgroup Impact 
Characteristic and Subgroup Size (%) (%) impact p* Difference p* 
Employed within past year -2.0 0.598 
No 914 35.0 19.6 15.4*** 0.000 oo o- 
Yes 1,027 39.9 22.5 17.4*** 0.000 - --- 
Prior employment, by gender - 0.952 
Women employed within 
past year 
No 583 34.6 18.7 15.9°** 0.000 oe --- 
Yes 458 42.2 24.6 17.6*** 0.000 - - 
Men employed within past year 
No 331 36.0 21.7 14,3°** 0.003 os ad 
Yes 569 37.8 20.8 17.0*** 0.000 - -- 
' Left school in grade 11 or 12 2.6 0.521 
= No 1,140 36.7 19.2 17.5*** 0.000 -- “- 
? Yes 801 38.9 24.0 14.9*** 0.000 - -- 
Received occupational training 
within past year 0.4 0.945 
No 1,615 37.8 21.3 16.5*** 0.000 - o- 
Yes 326 36.6 20.5 16,1*** 0.001 - --- 
Age 21 0.643 
16-19 1,425 38.2 22.3 15.9°** 0.000 one o-- 
20 or 21 516 36.1 18.1 18.0°** 0.000 - -- 
Age, by gender - 0.967 
Women 
16-19 763 39.0 23.1 15.9*** 0.000 one -- 
20 or 21 278 36.2 17.4 18,8*** 0.000 - -- 
Men 
16-19 662 37.2 21.3 15.9*%** 0.000 oe - 
20 or 21 238 35.9 19.0 16.9*** 0.003 — -- 
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TABLE 4.7 (continued) 


_—End of Month 48 __ 
Sample _ Experimentals Controls Subgroup 


Characteristic and Subgroup Size (9) (%) Impact 

Marital status 
Ever married 184 47.9 19.8 26,1*** 

Never married 36.6 21.3 18.3*** 

Living in own household or 

with boy/girlfriend 
No 16.7*** 

Yes 1s.aee* 

Own AFDC case or receiving 

General Assistance 
No 17.20e* 

Yes h 14,4*** 

Own AFDC case 
No 16.7*** 

Yes : 15.6*** 

Receiving Food Stamps 
No Y 16.5*** 

Yes 16.4*** 

Arrested since age 16 4.2 0.459 
No 1,649 37.8 22.0 15.6""* 0.000 o- = 
Yes 292 36.2 16.2 20.0°** 0.000 - wind 

Arrested since age 16, 

by gender - 0.781 
Women 

No 986 38.4 22.2 16.2*** 0.000 -- - 
1 92 Yes 55 45.2 17.9 27.3** 0.022 - = 
Men 
No 663 37.3 21.9 15.4*¢* 0.000 -_ one l fe) 3 
Yes 237 33.7 15.0 18.7*** 0.001 -- ‘ 


(continued) 
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TABLE 4.7 (continued) 


Received GED by 
End of Month 48 
Sample _ Experimentals = Controls Subgroup Subgroup Impact 
Characteristic and Subgroup Size (%) (%) Impact p* Difference p* 
Lived with both parents at 
age 14 44 0.281 
No 1,264 37.5 22.6 14.9*** 0.000 - - 
Yes 677 37.9 18.6 19.3%** 0.000 - - 
Reason for leaving regular 
high school - 0.958 
School-related 925 36.8 20.1 16.7*** 0.000 -- -- 
Job-related 197 43.6 25.6 18.0*** 0.004 - - 
Other 819 37.2 21.1 16.1*** 0.000 oe -- 


SOURCE: MDRC calculatiors from JOBSTART enrollment form and survey data. 


NOTES: Calculations for this table used data for all sample members for whom there were 48 months of follow-up survey data, 
including those with values of zero for outcomes and those who were assigned to JOBSTART but did not participate. 

Average experimental and control group outcomes reported here are adjusted means from two-way analysis of covariance 
procedures controlling for up to 31 kinds of difference in characteristics, other than the characteristic used to define subgroups, before 
random assignment. The two categories used as factors were research assignment and, one at a time, the baseline characteristics indicated 
(see Ostle, 1975, p. 454). There may be slight discrepancies in reported sums and differences of these adjusted means because of rounding. 

*A two-tailed t-test was applied to each within-subgroup impact and also to each difference between subgroup impacts. For each 
characteristic with more than two subgroups, an F-test was applied to the interaction between that characteristic and experimental or control 
status. The columns labeled "p* are the statistical significance levels of each impact and each difference in impacts or F-statistic: that is, p is 
the probability that sample estimates are non-zero only because of random error. Statistical significance levels are indicated as *** = 1 
percent; ** = 5 percent; * = 10 percent. 

>For each characteristic that has only two subgroups, the subgroup impact difference is the impact within the first subgroup, less the 
impact within the second subgroup. 
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sample, as evident in Table 4.7, which shows 48-month impacts on GED receipt.!7 For example, 
the first three rows of the table present impacts on GED receipt for women and men. For women, 
the impact was 16.7 percentage points, while for men it was 16.2 percentage points. The difference 
in impacts for these two groups, 0.5 percentage points, is reported in the column labeled “subgroup 
impact difference,” and is not statistically significant. Other subgroup findings include impacts of 
more than 15 percentage points for whites, blacks, and Hispanics; an impact of 17.5 percentage 
points for youths who left school in ninth or tenth grade, and 14.9 percentage points for those who 
left in eleventh or twelfth grade; similar impacts for those who left school for job-related (18 
percentage points) or school-related (16.7 percentage points) reasons, and for those with and 
without an arrest since age 16 (20 and 15.8 percentage points, respectively); and a 15.6 percentage 
point impact for youths with their own AFDC case at random assignment. 

JOBSTART did increase the educational attainment of young school dropouts, just as it 
increased their receipt of education and training services. However, many members of the 
experimental group remained without a GED or high school diploma at the end of the follow-up 
period. Together, these two major findings lead to an important question: Did JOBSTART — by 
providing additional education and training and increasing educational attainment — make a 
difference in the employment, earnings, and other key outcomes of youths? Chapters 5 and 6 
address this issue for the full sample and subgroups. 


1The fincings presented in this paragraph are based on an analysis similar to that used for Table 4.4: that 
is, impacts are for subgroups defined by a specific characteristic, with differences in the other observed 
characteristics statistically controlled for through linear regression. 
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CHAPTER 5 


IMPACTS ON EARNINGS AND EMPLOYMEN 


Chapter 4 showed that JOBSTART had strong impacts on the receipt of a high school diploma 
or GED certificate for most subgroups of the 48-month impact sample, that these impacts persisted 
over time, and that the vast majority of those who earned a high school diploma or GED had 
already done so by the end of the second year after random assignment. This chapter focuses on 
the extent to which the impact on educational attainment translated, in turn, into greater success 
in the labor market by the end of the fourth follow-up year. The findings on earnings and 
employment presented here are not the full story for the JOBSTART sample because the 
individuals are still in only their early tu mid-twenties. The findings are, however, the fina! results 
for the JOBSTART project because there are no plans to gather additional follow-up data. 

The following key questions about labor market outcomes are addressed for the full 48-month 
impact sample and for important subgroups: 

© What sacrifices of earnings or employment opportunities did experimentals make 


during the first year after random assignment, which for many was primarily a 
period spent in JOBSTART? 


¢ Had the experimentals begun to catch up with the controls in earnings and 
employment by the end of the second year? The third year? The fourth year? 


¢ Did the cumulative earnings impact turn positive at some point during the four 
years? If so, when? Did it continue to grow thereafter? 


* Do more precise measures of work effort (employment rates, hours worked, and 
wages earned per hour) shed additional light on the basic earnings impacts during 
the four-year follow-up period? 


* How did experimentals and controls compare in earnings and employment within 
important subgroups? Were the labor market effects of JOBSTART different for 
men, women who were custodial mothers, and other women? Did JOBSTART’s 
impacts on earnings vary according to age, grade at the time of dropout, reason for 
dropping out of school, criminal recorz, or other characteristics of the young 
people in the sample? 


The next chapter presents a similar analysis for other outcomes, such as public assistance 
receipt, childbearing, criminal arrests, and drug use, again with subanalyses for key subgroups. It 
concludes with a summary of all the measured impacts for the key subgroups. In both this chapter 
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and the next it is important to recognize that dividing the sample into smaller subgroups reduces the 
likelihood that the impact estimates will be statistically significant. We present the subgroup findings 
— despite their reduced statistical power — because of the strong interest in learning how the 
program worked for different groups within the JOBSTART population. 


I mpacts o in 


The overall impact of JOBSTART on the earnings of the 48-month impact sample was a 
statistically insignificant gain of $214, or 1.3 percent of what the experimentals would have earned 
over the four years after random assignment if they had not had access to JOBSTART. This overall 
impact is a weighted average of a small loss for men and gains of several hundred dollars each for 
both custodial mothers and all other women. None of these subgroup impacts was statistically 
significant. The time path of the estimated impacts indicates that all three groups made some 
sacrifices in earnings during the first, predominantly in-program, year; further sacrifices for men, but 
not for either group of women, during the second yeas, and small gains for all three groups during 
years three and four. 

Table 5.1 summarizes JOBSTART's impacts on earnings for the full 48-month impact sample. 
As shown in the first row, JOBSTART raised, by an average of $214, cumulative earnings among 
the 1,941 sample members (of the 2,311 at baseline) who had four years of continuous follow-up 
data. Specifically, control group members earned an average of $16,796 over the four-year span, 
while experimentals earned an average of $17,010, or 1.3 percent more than the control group 
mean.! The $214 impact was not statistically significant, however. The $214 estimated gain arose 
from positive impacts during the third and fourth years following random assignment of $423 and 
$410, respectively, which were partially offset by impacts of -$499 during the first year and -$121 
during the second year of follow-up. Only the first-year estimate was statistically significant. 

The three panels of Table 5.2 show the results of separate impact calculations for men, custodial 
mothers, and all other women. Although none of the estimated impacts on cumulative four-year 


earnings for the three subgroups was statistically significant, point estimates were positive for both 


1Note that the average: for the control group and the experimental group include zero earnings for those 
who did not work. The average for the experimentals includes data on those who had access to JOBSTART 
but did not participate, who made up 11 percent of all experimentals. Based on the assumption that 
nonparticipants were unaffected by JOBSTART or other programs, impacts on participants would be 12 
percent higher than the total reported in the text for all experimentals. See Appendix B for the details of this 
adjustment. 
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TABLE 5.1 
IMPACTS ON ANNUAL EARNINGS THROUGH MONTH 48 


Experimentals Controls Difference 
Outcome and Follow-Up Period ($) ($) ($) 


Total earnings 
17,010 16,796 214 


2,097 ~499*** 
3,991 -121 
5,329 423 
5,592 410 


988 953 


SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 


NOTES: Cajculations for this table used data for all sample members for whom there were 
48 months of follow-up survey data, including those with values of zero for outcomes and those who 
were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For some sample 
members, the month of random assignment may be less than a month, beginning with the date of 
random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reporied here are adjusted means from 
linear analysis of covariance procedures controlling for up to 31 kinds of difference in characteristics 
before random assignment (see Ostle, 1975, p. 461; Cave, 1987; and Appendix B). There may be 
slight discrepancies in reported sums and differenc?2s of these adjusted means because of rounding. 

*The column labeled ‘p’ is the statistical significance level of the difference between 
experimental and control group outcomes: that is, p is the probability that average outcomes are 
differem only because of random error. A two-tailed t-test was applied to each difference between 
average experimental and control group outcomes. Statistical significance levels are indicated as *** 
= 1 percent; ** = 5 percent; * = 10 percent. 
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TABLE 5.2 


IMPACTS ON ANNUAL EARNINGS THROUGH MONTH 48, 
BY GENDER AND PARENTAL STATUS 


Subgroup, Outcome, 


and Follow-Up Period Experimentals Controls Difference p* 
Men 
Total earnings ($) 
Years 1-4 23,364 23,637 -273 0.818 
Year 1 2,929 3,741 -812*** 0.003 
Year 2 5,435 5,831 -396 0.266 
Year 3 7,401 6,957 444 0.320 
Year 4 7,599 7,107 492 0.271 
Sample size 448 452 


Women living with own child(ren) 
Tota! earnings ($) 


Years 1-4 8,959 8,334 625 0.557 

Year 1 1,016 1,160 -144 0.451 

Year 2 2,097 1,947 150 0.648 

Year 3 2,700 2,572 328 0.409 

Year 4 3,146 2,855 290 0.505 
Sample size 257 251 


Women not living with own child(ren), 
including those who did not have any 


Total earnings ($) 


Years 1-4 13,923 13,310 613 0.609 

Year 1 1,697 2,040 343 0.126 

Year 2 3,345 3,269 76 0.831 

Year 3 4,309 3,889 420 0.360 

Year 4 4,572 4,111 451 0.342 
Sample size 283 250 


SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 


NOTES: Calculations for this table used data for all sample members for whom there were 
48 months of follow-up survey data, including those with values of zero for outcomes and those who 
were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For some sample 
members, the month of random assignment may be less than a month, beginning with the date of 
random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted means from split- 
file linear analysis of covariance procedures controlling for up to 29 kinds of difference in 
characteristics before random assignment (see Ostle, 1975, p. 461; Cave, 1987; and Appendix B). 
There may be slight discrepancies in reported sums and differences of these adjusted means 
because of rounding. 

*The column labeled °p’ is the statistical significance level of the difference between 
experimental and control group outcomes: that is, p is the probability that average outcomes are 
different only because of random error. A two-tailed t-test was applied to each difference between 
average experimental and control group outcomes. Statistical significance levels are indicated as *** 
= 1 percent; ** = 5 percent; * = 10 percent. 
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groups of women but negative for men. Custodial mothers in the experimental group gained $625, 
or 7.5 percent of their control group's mean of $8,334. Other women in the experimental group 
gained $613, or 4.6 percent of their control group’s mean of $13,310. And the men lost $273, or 
-1.2 percent of their control group’s mean of $23,637. 

For all three groups, the impact estimates were negative during the first, predominantly 
in-program, year. The custodial mothers sacrificed $144 during that first year, other women 
sacrificed $343, and men gave up $812 in earnings. These differences in initial impacts in part 
reflect the sharp differences in earnings across the control groups; men in the control group earned 
much more than women who were not custodial mothers, who in turn earned considerably more 
than custodial mothers. Although the experimental-contro! difference in earnings remained negative 
for men during the second year, it turned slightly positive for both groups of women during that 
time. By the third year, experimental-control differences were positive for all three groups, and they 
remained positive during the fourth year. These statistically insignificant differences fell slightly in 
magnitude from year three to year four for custodial mothers, but grew slightly for men and other 
women. During the final year of follow-up, the experimental-control differences in earnings were 
$290 (or 10.2 percent) for custodial mothers, $461 (11.2 percent) for other women, and $492 (6.9 
percent) for men. 

Table 5.3 presents earnings impact estimates for male sample members arrested between age 
16 and random assignment and for all youths in the sample according to their reason for dropping 
out of regular high school. Impacts for males arrested since age 16 (shown in the top panel of the 
table) were positive in years two through four of follow-up and for the full four years, with 
statistically significant impacts in the fourth year despite the small sample size. Although the small 
sample size of this group means that caution must be exercised in interpreting the subgroup impact 
findings, it does appear that JOBSTART had noticeably better impacts for young males with a prior 
arrest than other young males, for whom overall earnings impacts remained negative at the four-year 
point. 

JOBSTART's strong positive impacts for young males arrested since age 16 are in stark 
contrast to findings from the National JTPA Study at the 18-month point. In that study, the 
impacts of JTPA-funded Title ITA services were noticeably worse for young out-of-school males with 


2The impacts for all women (combining the two female subgroups) were also statistically insignificant. 
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TABLE 5.3 


IMPACTS ON ANNUAL EARNINGS THROUGH MONTH 48, 


FOR KEY SUBGROUPS 
Subgroup, Outcome, 
and Follow-Up Period Experimentals Controls Difference p* 
Men arrested since age 16 
Total earnings ($) 
Years 1-4 22,835 20,344 2,491 0.313 
Year 1 3,091 4,027 -936 0.110 
Year 2 5,722 5,297 425 0.605 
Year 3 7,052 5,923 1,129 0.187 
Year 4 6,970 5,098 1,872** 0.039 
Sample size 127 110 
Full sample, by reason for 
leaving regular high school 
School-related 
Total earnings ($) 
Years 1-4 17,590 16,409 1,181 0.250 
Year 1 2,179 2,491 312 0.157 
Year 2 4,287 4,112 175 0.579 
Year 3 5,486 4,760 726* 0.054 
Year 4 5,638 5,046 592 0.131 
Sample size 489 436 
Job-related 
Total earnings ($) 
Years 1-4 27,585 27,815 -230 0.933 
Year 1 3,723 4,831 -1,108 0.101 
Year 2 6,672 6,452 220 0.801 
Year 3 8,442 8,188 254 0.806 
Year 4 8,746 8,343 405 0.719 
Sample size 91 106 
Other 
Total earnings ($) 
Years 1-4 13,871 14,449 -578 0.556 
Year 1 1,611 2,156 -545*** 0.005 
Year 2 3,086 3,461 -375 0.192 
Year 3 4,405 4,258 147 0.704 
Year 4 4,770 4,574 196 0.617 


Sample size 


(continued) 


TABLE 5.3 (continued) 


SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 


NOTES: Calculations for this table used data for all sample members for whom there were 
48 months of follow-up survey data, including those with values of zero for outcomes and those who 
were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For some sample 
members, the month of random assignment may be less than a month, beginning with the date of 
random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted means from split- 
file linear analysis of covariance procedures controlling for up to 31 kinds of difference in 
characteristics before random assignment (see Ostle, 1975, p. 461; Cave, 1987; and Appendix B). 
There may be slight discrepancies in reported sums and differences of these adjusted means 
because of rounding. 

*The column labeled ‘p* is the statistical significance level of the difference between 
experimental and control group outcomes: that is, p is the probability that average outcomes are 
different only because of random error. A two-tailed t-test was applied to each difference between 
average experimental and control group outcomes. Statistical significance levels are indicated as *** 
= 1 percent; ** = 5 percent; * = 10 percent. 
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a prior arrest than for other young out-of-school males, a finding that is consistent with the apparent 
pattern of worse impacts for male youths who face more barriers to employment.? 

Comparing the earnings impacts for subgroups based on reasons for dropping out of high 
schooi, those for youths who had dropped out for school-related reasons were the best. These 


youths had the lowest initial earnings losses among the three subgroups, a statistically significant 


earnings impact of $726 in the third year of follow-up, and a four-year impact of $1,181. In contrast, 
those who had dropped out of high school to find a job (and whose control group counterparts were 
most likely to be working) had large earnings losses in the first year and relatively small gains in 
subsequent years, for a four-year impact that was slightly negative. Those who had dropped out for 
other reasons fared the worst over the four years: They had high initial earnings losses and the 
smallest earnings gains in years three and four. 


Il. The Timing of Payoffs to the Investment in JOBSTART 


As is true of most investments, many of the costs of participating in a program such as 
JOBSTART are incurred in the short run. Any benefits will accrue over the rest of the participant's 
lifetime, if the new skills learned pay off in the labor market. Figure 5.1 presents a theoretical 
model for analyzing the labor market aspects of this investment and an alternative investment made 
by young people in the control group. 

The first shaded area at the left of the figure represents foregone earnings sacrificed by 
program participants while they expend time and effort to iinprove their skills, in hope of a future 
payoff. These foregone earnings are an example of what economists term the "opportunity costs" 
of choosing one course of action over another.‘ 

The figure also predicts that the control group's earnings will rise over time, as the group's 
employment rate rises with the group’s growing maturity and experience and, for some members, 
with their acquisition of new skills on the job. For young people with poor skills, work experience 
can be an important source of new skills, which can translate into increased productivity and 
earnings and more stable employment. For their part, the young people in the experimental group 
can look for a job after their participation in JOBSTART ends, but their employment rates and 
earnings may not immediately exceed (or even reach) those of the controls who were already 


3See Bloom et al., 1993. 
‘in our example, these opportunity costs could also include other costs of program participation over and 
above the amount of foregone earnings. 
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working rather than attending a program. This post-program opportunity cost is the area in the 


figure between the end of program participation and the hoped-for point at which the earnings of 
the experimental group exceed those of the control group. 

For a program such as JOBSTART to pay off for young people, the long-term benefits of 
increased education and training (represented in the figure by the shaded area on the right marked 
“payoff") must exceed the foregone — and more immediate — rewards of possible earnings and 
enhanced skills through work experience (represented in the figure as “opportunity costs”). Even 
in successful programs, it will take time for participants to overcome the head start of those who 
have been working throughout the program period. In other words, the "crossover" point (when 
experimentals start to earn more than controls) will not occur immediately.° Eventually, however, 
for the program to benefit participants, the amount of payoff must exceed the amount of 
Opportunity cost (the shaded area on the right must become larger than the shaded area on the 
left).© Further, from a societal perspective, for a program to pass a test of economic efficiency, the 
benefits created by the program (for example, in the form of increased earnings) must exceed the 
resources used to provide program services. 

The evaluation of the residential Job Corps program cited in Chapter 1 provides an example 
of an investment in education and training that was shown to benefit both participants and society.’ 
That study began its follow-up at the point of termination from the program, and it reported impacts 
only for six-month intervals rather than for individual months. For the period from program 
termination to six months thereafter, the employment and earnings of those in the program group 
(participants) were slightly lower than those of the comparison group (although the differences were 
not statistically significant). The employment rates and earnings of the program group began to 
exceed those of the comparison group 6 to 12 months following program termination, and they 
continued to be higher throughout the remainder of the 48-month follow-up period. Moreover, the 
cumulative earnings gains the participants experienced over the follow-up period — the benefits — 
exceeded the costs of the program.® 


5A more detailed analysis of how JOBSTART affected the income of youths would include changes in 
other sources of income, such as public assistance, for experimentals and controls. 
®As discussed in Chapter 7, benefits and costs must be discounted properly so they can be compared. 
7See Mallar et al., 1982, p. 135. 
findings are not directly applicable to the JOBSTART evaluation because follow-up in JOBSTART 
began at random assignment, the point when youths were ready to enter the program. However, since the 
average length of stay in JOBSTART was roughly six months, the 12-month survey was conducted about six 
months after the average experimental group member left the program, and the 24-month survey was conducted 
(continued...) 
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The findings presented in Chapters 3 and 4 described the initial period of investment in 
JOBSTART services. During the first year after random assignment, JOBSTART experimentals had 
less time available for work than did their control group counterparts, because for much of that 
period they were in an intensive program of education ai * skills instruction. Chapter 3 showed that 
more than SO percent of experimentals were still active in the sixth month after entering the sample; 
15 percent were still active in the twelfth month; and almost 10 percent were still active in the 
fifteenth month. Throughout most of the first two years of follow-up, a larger percentage of 
experimentals than controls were participating in an education or training program. The controls, 
therefore, got a head start in the labor market, since the experimentals could not both be in 
JOBSTART and be working during any given hour of the day. (Again, for young people with poor 
skills, work experience itself can be an important source of new job skills and higher wages.) GED 
attainment during or after intensive JOBSTART education ultimately may open up many 
empioyment opportunities for JOBSTART graduates, but even after an experimental group member 
left JOBSTART, it still might have taken some time to become as well-settled in the labor market 
as his or her control group counterpart, who might have been learning new job skills while working. 
Thus, well into the second year of follow-up, controls still might be expected to have had the edge 
on experimentals in employment and earnings. 

How well does the actual time path of earnings fit the theoretical model of investment and 
payoff? Figures 5.2a through 5.2d give the answer to this question by disaggregating the annual 
estimates of Tables 5.1 and 5.2 into monthly earnings for experimentals and controls. Figure 5.2a 
charts the progress of the full sample, while 5.2b through 5.2d present the story for custodial 
mothers, all other women, and men. ARhough none of the graphs look precisely like the theoretical 
model in Figure 5.1, there is a broad resemblance to the main features of the model in each of the 
four cases: a period of investment followed by a payback period. The point at which the curve for 
experimentals crosses above the curve for controls is near month 23 for the full impact sample, 
month 24 for the subgroup of men, month 25 for custodial mothers, and month 20 for other women. 

In the discussion of JOBSTART’s benefits and costs in Chapter 7, benefits other than earnings 
gains are also considered. But this examination of trends suggests that JOBSTART's impacts on 
earnings pass the first test for a successful investment: A payoff seems to have begun for many key 


8(...continued) 
about 18 months after that average experimental left the program. Thus, the semi-annual earnings and 
employment impacts in the Job Corps study turned positive at a point roughly corresponding to the year-two 
estimates in this report. 
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FIGURE 5.2a 
MONTHLY EARNINGS FOR THE FULL SAMPLE, BY RESEARCH STATUS 


Total Eamings Per Month ($) 
8 § 


2 4 6 8 10 12 14 16 18 20 22 24 26 26 W 32 34 3% 38 40 42 44 46 46 
Months Since Random Assignment 


SOURCE: Appendix Table D.1. 

NOTES: Calculations for this figure used data for all sample members for whom there 
were 48 months of follow-up survey data, including those with values of zero for oulcomes and those 
who were assigned to JOBSTART but did not participate. 

Random assignment did not always take piace on the first of the month. For some sample members, 
the month of random assignment may be less than a month, beginning with the date of random assignment 
and ending on the last day of the month. 
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FIGURE 5.2b 


MONTHLY EARNINGS FOR WOMEN LIVING WITH THEIR OWN CHILD(REN), 
BY RESEARCH STATUS 


Total Eamings Per Month ($) 
s 3 


8 


2 4 & 8 10 12 14 16 18 20 22 24 26 26 30 32 34 36 38 40 42 44 46 48 
Months Since Random Assignment 


SOURCE: Appendix Table D.2. 


NOTES: Calculations for this figure used data for all sample members for whom there 
were 48 months of follow-up survey data, including those with values of zero for outcomes and those 
who were assigned to JOBSTART but did not participate. 
Random assignment did not always take place on the first of the month. For some sample members, 


the month of random assignment may be less than a month, beginning with the date of random assignment 
and ending on the last day of the month. 
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FIGURE 5.2c 


MONTHLY EARNINGS FOR WOMEN NOT LIVING WITH THEIR OWN CHILD(REN), 
BY RESEARCH STATUS ® 


Total Eamings Per Month ($) 


2 4 6 8 10 12 14 16 18 20 22 24 26 26 30 32 34 36 38 40 42 44 46 48 
Months Since Random Assignment 


SOURCE: Appendix Table 0.3. 


NOTES: Calculations for this figure used data for all sample members for whom there 
were 48 months of follow-up survey data, including those with values of zero for outcomes and those 
who were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For some sample members, 
the month of random assignment may be less than a month, beginning with the date of random assignment 
and ending on the last day of the month. 

® includes women who did not have children. 


FIGURE 5.2d 
MONTHLY EARNINGS FOR MEN, BY RESEARCH STATUS 


Experimentals 
Controls 


Total Earnings Per Month ($) 
8 g 68 
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Months Since Random Assignment 


SOURCE: Appendix Table 0.4. 


NOTES: Calculations for this figure used data for all sample members for whom there 
were 48 months of follow-up survey data, including those with values of zero for oulcomes and those 
who were assigned to JOBSTART but did not participate. 
Random assignment did not always take place on the first of the month. For some sample members, 


the month of random assignment may be less than a month, beginning with the date of random assignment 
and ending on the last day of the month. 
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subgroups before the end of the follow-up period. Moreover, Tables 5.2 and 5.3 show that for some 
subgroups — custodial mothers, other women, males arrested since age 16, and youths who dropped 
out of school for school-related reasons - JOBSTART passes the second test as of the four-year 
point: Earnings gains in the later months of follow-up outweighed the initial foregone earnings so 
that the experimentals appeared to earn more than the controls (although the four-year impacts 
were not statistically significant). Chapter 7 discusses whether JOBSTART passes another efficiency 
test by having a sufficiently large excess of payoffs (both in carnings gains and other benefits) to 
cover direct program and other costs. 


Ill. lo of Wo d W Wor 


A different kind of analysis of earnings entails looking beyond time paths of monthly earnings 
to the impacts of JOBSTART on the components of earnings. More specifically, earnings are the 
product of hours worked and wages per hour. Thus, any percentage change in earnings is 
approximately equal to the sum of a percentage change in hours of work and a percentage change 
in earnings per hour. Moreover, hours of work can change for a number of reasons, including 
changed rates of employment, changed hours of work per week, and changed weeks of work per 
period. Analyzing these more refined measures of labor market success helps explain the basic 
findings on earnings impacts. This section examines differences between experimentals and controls 
in the time spent working; the following section will discuss differences in wages between those 
experimentals and controls who worked ° 

As expected, controls were more likely to work than experimentals in the first year of follow- 
up. In the second year, despite a slightly higher employment rate for experimentals (71 percent) 
than for controls (67.5 percent), experimentals were not ahead in actual time spent working, 
although there were differences among the key subgroups. During the latter part of the four-year 
follow-up, experimentals began to receive the payoff for their earlier investment in human capital. 
This and the next section cover these more refined measures of investment and the beginning of the 
payoff for the full sample and each of the three major subgroups. 

Because young people tend to apply for employment and training programs when they are 
between jobs or about to enter or reenter the labor force, the trend in the monthly employment rate 


*The separate presentations are meant to highlight an important methodological distinction: The impacts 
on employment rates, hours worked, and weeks worked are estimated experimentally, whereas the impacts on 
wage rates for those who worked are estimated using nonexperimental techniques. 
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was upward during the first two years for both experimentals and controls (Figure 5.3a). For the 
full sample, a larger fraction of controls than experimentals was employed in each month of the first 
year. The difference in employment rates peaked in months four and five and then narrowed. After 
month 12, the average employment rate of experimentals was greater than that of controls more 
often than it was below it, but in only one month was the differe:ce statistically significant. During 
the last two years of follow-up, monthly employment rates leveled off in the middle to high 40 
percent range, with experimentals mainly employed at monthly rates insignificantly higher than those 
for controls (only during month 37 was the difference statistically significant). Figures 5.3b and 5.3c 
show that the average number of hours and weeks worked per month followed roughly the same 
pattern as employment rates. 

Table 5.4 aggregates the monthly results in Figure 5.3a into cumulative and annual outcomes 
and impacts for the full sample. Over the four-year period as a whole there was almost no 
experimental-control difference in the percentage employed at some time: 86.4 percent of 
experimentals and 86 percent of controls were employed at some point. The modest 1.3 percent 
gain in four-year earnings therefore must have come mainly from greater earnings per hour for 
experimentals (as discussed in Section IV below), since it does not seem to have stemmed from 
greater employment rates or more hours or weeks of work. The year-by-year pattern is instructive 
as well. The -$499 (19 percent) loss in year-one earnings stemmed from negative program impacts 
on all three employment measures: the employment rate and weeks and hours worked. During year 
two, the employment rate of experimentals exceeded that of controls, the disparity in weeks of work 
disappeared, and the hours gap narrowed to a very small amount, but not enough to yield earnings 
gains during the second year. During the last two years the earnings gains were mainly the result 
of more hours and weeks worked by experimentals and not of a difference in employment rates. 


A. Impacts for Custodial Mothers 


As explained in Chapter 4, many previous evaluations of employment and training programs 
for youths have found better program effects for young women than for young men.!° One likely 
explanation for women’s better employment results is that it is easier to improve the employment 
and earnings of those who do not spend much time in the world of work (for example, young 
mothers) than of those who are already in the labor force but fail to find and keep steady, 
well-paying jobs (for example, young men with poor skills). Thus, from this perspective, young 


19For a comprehensive review of the studies, see Betsey et al., 1985. 
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FIGURE 5.3a 
MONTHLY EMPLOYMENT RATES FOR THE FULL SAMPLE, BY RESEARCH STATUS 


Employed During Month (%) 
8 


2 4 6 8 10 12 14 16 18 20 22 24 26 26 30 32 34 36 38 40 42 44 46 48 
Months Since Random Assignment 


SOURCE: Appendix Table D.5. 
NOTES: Calculations for this figure used data for all sample members for whom there 
were 48 months of follow-up survey data, including those with values of zero for outcomes and those 
who were assigned to JOBSTART but did not participate. 
Random assignment did not always take place on the first of the month. For some sample members, 


the month of random assignment may be less than a month, beginning with the date of random assignment 
and ending on the last day of the month. 
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FIGURE 5.3b 
MONTHLY HOURS WORKED FOR THE FULL SAMPLE, BY RESEARCH STATUS 


2 4 6 8 10 12 14 16 18 20 22 24 26 26 30 32 34 36 38 40 42 44 46 48 
Months Since Random Assignment 


SOURCE: Appendix Table 0.6. 

NOTES: Calculations for this figure used data for all sample members for whom there 
were 48 months of follow-up survey data, including those with values of zero for outcomes and those 
who were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For some sample members, 
the month of random assignment may be less than a month, beginning with the date of random assignment 
and ending on the last day of the month. 


FIGURE 5.3c 
MONTHLY WEEKS WORKED FOR THE FULL SAMPLE, BY RESEARCH STATUS 


2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 
Months Since Random Assignment 


SOURCE: Appendix Table 0.7. 

NOTES: Calculations for this figure used data for all sample members for whom there 
were 48 months of follow-up survey data, including those with values of zero for outcomes and those 
who were assigned to JOBSTART but did not participate. 

Random assignment did not always take piace on the first of the month. For some sample members, 
the month of random assignment may be less than a month, beginning with the date of random assignment 
and ending on the last day of the month. 
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TABLE 5.4 


IMPACTS ON ANNUAL EARNINGS, EMPLOYMENT RATES, 
HOURS OF WORK, AND WEEKS OF WORK 


THROUGH MONTH 48 
Follow-Up Period and Outcome Experimentals Controls Difference p* 
Years 1-4 
Total earnings (S$) 17,010 16,796 214 0.757 
Ever employed (%) 86.4 86.0 0.4 0.789 
Total hours worked 3,030.57 3,070.69 40.12 0.700 
Total weeks employed 79.96 80.81 -0.84 0.741 
Year 1 
Total (S) 2,097 2,596 -499°** 0.001 
Ever employed (9%) 56.5 60.8 -4.3** 0.041 
Total hours worked 441.08 550.13 -109.05*** 0.000 
Total weeks employed 12.67 15.30 -2.63°** 0.000 
Year 2 
Total earnings ($) 3,991 4,112 -121 0.563 
Ever employed (%) 71.0 67.5 3.5* 0.073 
Total hours worked 760.23 775.30 -15.06 0.660 
Total weeks employed 20.32 20.27 0.05 0.953 
Year 3 
Total earnings ($) 5,329 4,906 423 0.102 
Ever employed (%) 61.8 61.5 0.3 0.865 
Total hours worked 898.96 854.83 44.12 0.249 
Total weeks employed 23.20 22.28 0.92 0.323 
Year 4 . 
Total ($) §,592 §,182 410 0.125 
Ever employed (%) 65.7 64.5 13 0.836 
Total hours worked 930.30 890.42 39.88 0.301 
Total weeks employed 23.77 22.95 0.82 0.370 
Sample size 988 953 


SOURCE: MDRC calculations from JOBSTART enroliment form and survey data. 


NOTES: Calculations for this table used data for all sample members for whom there were 
48 months of follow-up survey data, including those with values of zero for outcomes and those who 
pacicke lence linatiaichepan gteneniaeaite 
did not always take piace on the first of the month. For some sample 
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Average experimental and control group outcomes reported here are adjusted means from 
controlling 
before random assignment (see Ostle, 1975, p. 461; Cave, 1987; and Appendix B). There may be 


slight discrepancies in reported sums and differences of these adjusted means because of rounding 
*The column labeled ‘p’ is the statistical significance level of the difference between 


women have greater potential for improved labor market outcomes than young men, and less to lose 
(in terms of foregone employment and earnings) by investing in education and training. The 
JOBSTART sample exemplifies this pattern.!! 

We begin with the findings for custodial mothers. Figures 5.4a-c display experimental-control 
differences in the monthly time paths of employment rates, hours worked, and weeks worked for 
these women. The graphs each show roughly the same time trend as the trend in monthly 
employment rates presented earlier in Figure 5.3a. Table 5.5 aggregates the monthly results for 
custodial mothers from Figures 5.4a-c into annual outcomes and impacts. As shown in the top 
panel, there was a statistically insignificant difference of 4.5 percentage points in the proportion who 
had any paid employment during the four-year follow-up period — 75.4 percent of experimentals 
versus 71 percent of controls. Thus, the $625 (7.5 percent) gain in four-year earnings for custodial 
mothers may have come primarily from higher employment rates for experimentals than for controls, 
since the overall gains in hours and weeks of work over the four years were noticeably smaller than 
7.5 percent. The year-by-year pattern in the last four panels shows a small but statistically 
insignificant gain in the employment rate even in the first year, a relatively large and significant 
employment rate gain during the second year, but a narrowing of the gap to virtually zero by the 
fourth year as the control group employment rate caught up and the experimental rate leveled off. 
In other words, by the fourth year, custodial mothers in the experimental group earned more than 
their control group counterparts primarily because they worked for more hours and weeks, not 
because they had higher employment rates or earnings per hour (as explained further in Section IV 
below). 


B. Impacts for Other Women 


Figures 5.5a-c show monthly time paths of employment rates, hours worked, and weeks worked 
for women who were not custodial mothers at baseline. The pattern here is quite different from 
that for custodial mothers. Employment rate differences seem to have been more important for 
other women than for custodial mothers, especially toward the beginning of follow-up (when 
controls worked more than experimentals) and toward the end (when experimentals moved ahead 
of controls in most months). When considering the control group's employment rates for other 
women in the early months of follow-up, it seems that experimentals sacrificed more hours and 


Mew of the impact estimates reported in this section are statistically significant, in part because of the 
relatively small sample size of the subgroups examined. 
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FIGURE 5.4a 


MONTHLY EMPLOYMENT RATES FOR WOMEN LIVING WITH THEIR OWN CHILD(REN), 
BY RESEARCH STATUS 


2 4 6 8 10 12 14 16 18 20 22 24 26 26 30 32 34 36 36 40 42 44 46 48 
Months Since Random Assignment 


SOURCE: Appendix Table 0.8. 

NOTES: Calculations for this figure used data for all sample members for whom there 
were 48 months of follow-up survey data, including those with values of zero for outcomes and those 
who were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For some sample members, 
the month of random assignment may be less than a month, beginning with the date of random assignment 
and ending on the last day of the month. 
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FIGURE 5.4b 


MONTHLY HOURS WORKED FOR WOMEN LIVING WITH THEIR OWN CHILD(REN), 
BY RESEARCH STATUS 


Experimentals 


2 4 6 8 10 12 14 16 18 20 22 24 26 26 30 32 34 36 3B 40 42 44 46 46 
Months Since Random Assignment 


SOURCE: Appendix Table D.9. 
NOTES: Calculations for this figure used data for all sample members for whom there 
were 48 months of follow-up survey data, including those with values of zero for oulcomes and those 


who were assigned to JOBSTART but did not participate. 
Random assignment did not always take place on the first of the month. For some sample members, 


the month of random assignment may be less than a month, beginning with the date of random assignment 
and ending on the last day of the month. 
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FIGURE 5.4c 


MONTHLY WEEKS WORKED FOR WOMEN LIVING WITH THEIR OWN CHILD(REN), 
BY RESEARCH STATUS 


2 4 6 8 10 12 14 16 18 20 22 24 26 26 W% 32 34 36 3B 40 42 44 46 48 
Months Since Random Assignment 


SOURCE: Appendix Table 0.10. 

NOTES: Calculations for this figure used data for all sample members for whom there 
were 48 months of follow-up survey data, including those with values of zero for outcomes and those 
who were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For some sample members, 
the month of random assignment may be less than a month, beginning with the date of random assignment 
and ending on the last day of the month. 
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TABLE 5.5 


IMPACTS ON ANNUAL EARNINGS, EMPLOYMENT RATES, HOURS OF WORK, 
ANO WEEKS OF WORK THROUGH MONTH 48, 
FOR WOMEN LIVING WITH THEIR OWN CHILD(REN) 
AT THE TIME OF RANDOM ASSIGNMENT 


Follow-Up Period and Outcome Experimentals Controls Difference p* 
Years 1-4 
Total earnings ($) 8,959 8,334 625 0.557 
Ever employed (%) 75.4 71.0 4.5 0.257 
Total hours worked 1,708.78 1,635.57 73.21 0.678 
Total weeks employed 47.45 46.29 1.16 0.798 
Year 1 
Total earnings ($) 1,016 1,160 -144 0.451 
Ever employed (%) 41.0 38.8 22 0.616 
Total hours worked 246.27 279.00 32.73 0.456 
Total weeks employed 7.49 8.26 0.77 0.511 
Year 2 
Total earnings ($) 2,097 1,947 150 0.648 
Ever employed (%) 53.2 45.5 7.8° 0.078 
Total hours worked 408.09 401.15 6.94 0.902 
Total weeks employed 11.72 11.26 0.46 0.758 
Year 3 
Total earnings ($) 2,700 2,372 328 0.409 
Ever employed (%) 42.7 41.2 1.5 0.740 
Total hours worked 505.96 456.54 49.42 0.456 
Total weeks employed 13.17 12.85 0.32 0.851 
Year 4 
Total 3,146 2,856 290 0.505 
Ever employed (%) 49.1 49.3 0.2 0.957 
Total hours worked 548.46 498.87 49.59 0.453 
Total weeks employed 15.08 13.92 1.16 0.490 
Sample size 257 251 


slight discrepancies in reported sums and differences of these adjusted means because of rounding. 
*The column labeled ‘p’ is the statistical significance level of the difference between 


FIGURE 5.5a 


MONTHLY EMPLOYMENT RATES FOR WOMEN NOT LIVING WITH THEIR OWN CHILD(REN), 
BY RESEARCH STATUS ° 


Employed a Month (%) 


2 4 6 B 10 12 14 16 18 20 22 24 26 28 30 32 34 96 36 40 42 44 46 48 
Months Since Random Assignment 


SOURCE: Appendix Table D.11. 

NOTES: Calculations for this figure used data for all sample members for whom there 
were 48 months of follow-up survey data, including those with values of zero for outcomes and those 
who were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For some sample members, 
the month of random assignment may be less than a month, beginning with the date of random assignment 
and ending on the last day of the month. 

® includes women who did not have children. 
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FIGURE 5.5b 


MONTHLY HOURS WORKED FOR WOMEN NOT LIVING WITH THEIR OWN CHILD(REN), 
BY RESEARCH STATUS ® 


2 4 6 8 10 12 14 16 18 20 22 24 26 26 30 32 34 36 36 40 42 44 46 48 
Months Since Random Assignment 


SOURCE: Appendix Table 0.12. 

NOTES: Calculations for this figure used data for all sample members for whom there 
were 48 months of follow-up survey data, including those with values of zero for outcomes and those 
who were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For some sample members, 
the month of random assignment may be less than a month, beginning with the date of random assignment 
and ending on the last day of the month. 

8 includes women who did not have children. 
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FIGURE 5.5c 


MONTHLY WEEKS WORKED FOR WOMEN NOT LIVING WITH THEIR OWN CHILD(REN), 
BY RESEARCH STATUS® 


2 4 6 86 10 12 14 16 18 20 22 24 26 26 390 S2 34 36 38 40 42 44 46 48 
Months Since Random Assignment 


SOURCE: Appendix Table 0.13. 

NOTES: Calculations for this figure used data for all sample members for whom there 
were 48 months of follow-up survey data, including those with values of zero for outcomes and those 
who were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For some sample members, 
the month of random assignment may be less than a month, beginning with the date of random assignment 
and ending on the last day of the month. 

®includes women who did not have children. 
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weeks of work to take part in the program than did their custodial mother counterparts, but they 
overtook the controls in these measures in later months, during the post-program period. 

Table 5.6 aggregates the other women’s monthly results from Figures 5.5a-c into annual 
outcomes and impacts. Over the full four-year period, there was no gain in employment rates — 
84.3 percent of experimentals and 85.3 percent of controls were employed at some point. While the 
dollar amount of earnings gains over the four years for other women was similar in magnitude to 
that for custodial mothers, it was a smaller percentage; in the control group, custodial mothers 
earned an average of $8,334 over four years, while other women earned an average of $13,310. In 
the third and fourth years of follow-up, gains in weeks of work seem important in explaining 
earnings gains for other women.!2 


C. Impacts for Men 


Figures 5.6a-c show much less of a difference over time in the monthly employment measures 
of male experimentals and controls than was the case for either custodial mothers or other women. 
After initial negative impacts on employment rates, hours worked, and weeks worked early in the 
follow-up period (when most experimentals were participating in JOBSTART), no positive and 
statistically significant monthly impacts on any of these outcomes appeared for men. This was in 
contrast to the trend for women who were not custodial mothers, for whom positive impacts did 
appear in the later months of follow-up. 

Table 5.7 aggregates the monthly results in Figures 5.6a-c into annual outcomes and impacts 
for young men. Over the full four-year period, almost all men worked at some point, and there was 
almost no difference between experimentals and controls in the percentage who ever worked — 94.1 
percent versus 94.5 percent. Behind the modest $273 (1.2 percent) loss in four-year earnings are 
larger percentage declines in hours and weeks of work. Average earnings per hour worked appear 
to have been higher for experimentals, thereby partially offsetting the negative impacts on hours and 
weeks of work. This issue is explored in more detail in the next section. 

During the first year after random assignment, 74.9 percent of male controls worked, compared 
to 38.8 percent of control group custodial mothers and 57.8 percent of other women. During this 
period, which for many experimentals was largely a time of program participation, the impact of 
JOBSTART on employment rates was 2.2 percentage points for custodial mothers, -2.2 percentage 


2Section IV presents nonexperimental estimates of impacts on hourly wages that suggest that wage gains 
were also important in the third and, especially, fourth years. 
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TABLE 5.6 


IMPACTS ON ANNUAL EARNINGS, EMPLOYMENT RATES, HOURS OF WORK, 
AND WEEKS OF WORK THROUGH MONTH 48, 
FOR WOMEN NOT LIVING WITH THEIR OWN CHILD(REN) 
AT THE TIME OF RANDOM ASSIGNMENT 


Follow-Up Period and Outcome Experimentals Controls Difference 


Years 1-4 
Total earnings ($) 13,923 13,310 613 
Ever employed (%) 84.3 85.3 -1.0 
Total hours worked 2,628.16 2,552.21 75.95 
Total weeks employed 72.64 67.94 4.70 
Year 1 
Total earnings ($) 1,697 
Ever employed (%) 
Total hours worked 
Total weeks employed 


Year 2 
Total earings ($) 
Ever employed (%) 
Total hours worked 
Total weeks employed 


Total weeks employed 21.19 19.15 2.04 0.268 
Sample size 283 250 


SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 


NOTES: Calculations for this table used data for all sample members for whom there were 
48 months of follow-up survey data, including those with values of zero for outcomes and those who 
were assigned to JOBSTART but did not participate. 

Random assignment did not always take piace on the first of the month. For some sample 


Average experimental and control group outcomes reported here are adjusted means from 
linear analysis of covariance procedures controlling for up to 28 kinds of difference in characteristics 
before random assignment (see Ostle, 1975, p. 461; Cave, 1987; and Appendix B). There may be 
slight discrepancies in reported sums and differences of these adjusted means because of rounding. 

*The column labeled ‘p* is the statistical significance level of the difference between 
experimental and control group outcomes: that is, p is the probability that average outcomes are 
differemt only because of random error. A two-tailed t-test was applied to each difference between 
average experimental and contro! group outcomes. Statistical significance levels are indicated as *** 
= 1 percent; ** = 5 percent; * = 10 percent. 
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FIGURE 5.6a 
MONTHLY EMPLOYMENT RATES FOR MEN, BY RESEARCH STATUS 


2 4 6 8 10 12 14 16 18 20 22 24 26 26 W 32 34 3% 3B 4 42 44 46 48 
Months Since Random Assignment 


SOURCE: Appendix Table D.14. 


NOTES: Calculations for this figure used data for all sample members for whom there 
were 48 months of follow-up survey data, including those with values of zero for outcomes and those 
who were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For some sample members, 
the month of random assignment may be less than a month, beginning with the date of random assignment 
and ending on the last day of the month. 
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FIGURE 5.6b 
MONTHLY HOURS WORKED FOR MEN, BY RESEARCH STATUS 


2 4 6 & 10 12 14 16 18 20 22 24 26 26 30 32 34 3% 38 40 42 44 46 48 
Months Since Random Assignment 


SOURCE: Appendix Table D.15. 

NOTES: Calculations for this figure used data for all sample members for whom there 
were 48 months of follow-up survey data, including those with values of zero for outcomes and those 
who were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For some sample members, 
the month of random assignment may be less than a month, beginning with the date of random assignment 
and ending on the last day of the month. 
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FIGURE 5.6c 
MONTHLY WEEKS WORKED FOR MEN, BY RESEARCH STATUS 


2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 
Months Since Random Assignment 


SOURCE: Appendix Table D.16. 

NOTES: Calculations for this figure used data for all sample members for whom there 
were 48 months of follow-up survey data, including those with values of zero for outcomes and those 
who were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For some sample members, 
the month of random assignment may be less than a month, beginning with the date of random assignment 
and ending on the last day of the month. 
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TABLE 5.7 


IMPACTS ON ANNUAL EARNINGS, EMPLOYMENT RATES, HOURS OF WORK, 
AND WEEKS OF WORK THROUGH MONTH 48, FOR MEN 


Follow-Up Period and Outcome Experimentals Controls Difference p* 
Years 1-4 
Total earnings ($) 23,364 23,637 -273 0.818 
Ever employed (%) 94.1 94.5 0.4 0.803 
Total hours worked 4,006.04 4,191.04 -185 0.267 
Total weeks employed 102.44 107.89 -5.44 0.164 
Year 1 
Total earnings ($) 2,929 3,741 812°" 0.003 
Ever employed (%) 65.7 74.9 -9,2*** 0.002 
Total hours worked §71.90 760.84 -188.95*** 0.000 
Total weeks employed 15.96 20.56 -4,.60°** 0.000 
Year 2 
Total earnings ($) 5,435 5,831 -396 0.266 
Ever employed (%) 83.0 82.3 0.7 0.772 
Total hours worked 1,003.82 1,067.47 -63.65 0.250 
Total weeks employed 26.08 27.19 1.11 0.397 
Year 3 
Total earnings (S$) 7,401 6,957 44a4 0.320 
Ever employed (%) 76.6 76.9 0.3 0.916 
Total hours worked 1,203.34 1,169.29 34.06 0.574 
Total weeks employed 30.14 29.95 0.19 0.894 
Year 4 
Total earnings ($) 7,599 7,107 492 0.271 
Ever employed (%) 78.6 76.7 2.0 0.468 
Total hours worked 1,226.99 1,193.44 33.55 0.583 
Total weeks employed 30.26 30.19 0.07 0.957 
Sample size 448 452 


SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 

NOTES: Calculations for this table used data for all sample members for whom there were 
Random assignment did not always take place on the first of the month. For some sample 
members, the month of random assignment may be less than a month, beginning with the date of 
month. 


slight discrepancies in reported sums and differences of these adjusted means because of rounding. 
*The column labeled "p’ is the statistical significance level of the difference between 


points for other women, and a statistically significant -9.2 percentage points for men. Again, these 
findings underscore how, even more for young men than for other women, JOBSTART entailed a 
substantial opportunity cost in foregone employment that custodial mothers did not have to pay. 
Even after four years, the hours and weeks of work forfeited during program participation had not 
been recouped in post-program gains. Clearly, this high opportunity cost is a primary cause of the 
poor results for men. 

A second possible explanation for the poorer impacts observed for men in JOBSTART is the 
greater difficulty of placing them in jobs that value a GED. Many women in JOBSTART, for 
example, were trained in clerical occupations and sought clerical jobs, for which educational 
credentials were important. However, young men were more likely to train for work that did not, 
at least initially, value a GED — for example, in many types of blue-collar occupations, especially 
physically demanding jobs.!> Further, it may have been harder to find training-related jobs for 
men who did study for occupations for which a GED mattered. 


IV. Impacts on Earnings per Hour 


Earnings are the product of hours spent working and wages earned per hour. Departing from 
the methodology of the analysis so far, the discussion in this section excludes those experimentals 
and controls who did not work during the periods indicated and therefore had no earnings. Thus, 
the experimental-control differences presented here are not experimentally estimated impacts, but 
they do help to understand the underlying factors of the impact findings.'* The focus continues 
to be on the three key subgroups: men, custodial mothers, and all other women. 

For each of thes: subgroups, both experimentals and controls generally showed gradual growth 
in earnings per hour over time, although the trends are not always clear. Table 5.8 shows the 
estimated hourly wages for working experimentals and controls for the three key subgroups. For 
custodial mothers, shown in the top panel of the table, estimated hourly wages for working 
experimentals were $4.14 in the first year and grew to about $5.45 by the fourth year; for working 
controls, hourly wages began at $4.00 and grew to $5.37. The differences in hourly wages in each 
year were relatively small, and during the entire four-year period, the average hourly wage for 


13Chapter 3 reported on occupations for which men and women trained and showed that a much higher 
perceatage of women studied for clerical occupations. 
experimental-control differences in this section were calculated using the Heckman correction for 
selection bias to account for the exclusion of experimentals and controls who did not work. See Heckman, 
1979. 
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ESTIMATES OF EFFECTS ON AVERAGE HOURLY WAGES THROUGH MONTH 48, 
BY GENDER AND PARENTAL STATUS 


Subgroup, Outcome, 
and Follow-Up Period 


Women living with own child(ren) 


Average hourly wages ($) 
Years 1-4 
Year 1 
Year 2 
Year 3 
Year 4 
Sample size (including nonworkers) 


Women not living with own child(ren), 
including those who did not have any 


Average hourly wages ($) 
Years 1-4 
Year 1 
Year 2 
Year 3 
Year 4 
Sample size (including nonworkers) 


Men 


Average hourly wages ($) 
Years 1-4 
Year 1 
Year 2 
Vear 3 
Year 4 


Sample size (including nonworkers) 


234 


194 
107 
137 
109 
125 
257 


TABLE 5.8 


Sample Size (Workers Only) 
Experimentals 


213 


Outcome 
Controls Experimentals Controls 


4.90 
4.14 
4.55 
5.54 
5.70 


5.45 
4.91 
5.13 
5.86 
5.89 


4.84 
4.00 
4.55 
5.09 
§.37 


4.82 
4.43 
4.91 
5.20 
4.93 


Difference 


-0.12 
0.14 
0.08 
0.01 
0.08 


0.08 
-0.29* 
-0.36 

0.34 

0.77*** 


0.09 
0.22 
-0.07 
0.03 
0.07 


0.526 
0.082 
0.111 
0.140 
0.008 


0.438 
0.421 
0.604 
0.869 
0.643 


(continued) 
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TABLE 5.8 (continued) 


SOURCE: MORC calculations from JOBSTART enroliment form and survey data. 


NOTES: Calculations for this table used data for all sample members for whom there were 48 months of follow-up survey data. 

Random assignment did not always take place on the first of the month. For some sample members, the month of random assignment 
may be less than a month, beginning with the date of random assignment and ending on the last day of the month. 

Average experimental und control group outcomes reported here are adjusted means from spilt-file linear analysis of covariance 
procedures controlling for uo to 29 kinds of difference in characteristics before random assignment (see Ostie, 1975, p. 461; Cave, 1987; and 
Appendix B). Only those sample members who worked during the follow-up period were included in these ANCOVA procedures. To correct 
for possible selection bias owing to the exclusion of nonworkers, inverse Mill's ratio terms were added, based on probit regressions using all 
sample members (see Heckman, 1979). There may be slight discrepancies in reported sums and differences of these adjusted means 
because of rounding. 

*The column labeled ‘p* is the statistical significance level of the difference between experimental and control group outcomes: that is, 
p is the probability that average outcomes are different only because of random error. A two-tailed t-test was applied to each difference 
between average experimental and control group outcomes. Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; 


* = 10 percent. 
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working controls ($4.84) exceeded that for experimentals ($4.72). For other women, the second 
panel in Table 5.8, hourly wages for working experimentals were below those of working controls 
in the first two years of follow-up, then moved well ahead in the third and fourth years. For men, 
the bottom pancl of the table, working experimentals earned slightly higher hourly wages than 
working controls in the first year, and earned nearly equal wages in succeeding years. 


V. Impacts for Other Selected Subgroups 


In addition to performing the split-sample analysis of earnings impacts presented above for the 
key subgroups, it is useful to estimate earnings impacts for subgroups controlling for differences in 
baseline characteristics other than that used to define the subgroups.!> Such an analysis helps to 
determine whether the impact for the subgroup continues to be present even after differences in 
other characteristics are controlled for, or whether these other characteristics may be correlated to 
the subgroup’s defining characteristic and have influenced the impact. This type of analysis also 
allows for a simpler determination of statistical significance for differences between subgroup 
impacts. 

Using this type of analysis, Tables 5.9 and 5.10 present both subgroup and (where applicable) 
between-subgroup impacts on earnings for the entire four-year follow-up period and for the final 
follow-up year, respectively. As in earlier tables of this type, in Chapter 4, impacts are presented 
for the subgroups listed in the left-hand column of the table. In the middle of the table, earnings 
for experimentals and controls are shown, and the column labeled "subgroup impact" presents the 
experimental-control difference in earnings for each subgroup. For example, in the first panel of 
Table 5.9, the impact for women was $654 and for men, -$298. Neither subgroup impact was 
statistically significant, as shown by the lack of stars and the high p-value. The “subgroup impact 
difference” column shows the difference between the individual subgroup impacts in cases where 
there are only two subgroups in the category. In the case of women and men, the difference was 
$952, which was not statistically significant, again indicated by the lack of stars and the high p-value. 
In the second panel (ethnicity), no individual subgroup impact was significant, though whites had 
a large positive impact and the "other" subgroup had a large negative impact. The differences 


'SThis approach was discussed in Chapter 2 and used in Chapter 4 to examine subgroup differences in 
educational attainment. 


-155- 


Characteristic and Subgroup 


Gender 
Women 
Men 


Ethnicity 
Waite, non-Hispanic 
Black, non-Hispanic 
Hispanic 
Other 


Ethnicity, by gender 
Women 


i 
: 
i 


Men 
White, non-Hispanic 
Black, non-Hispanic 
Hispanic 
Other 


Parental status 
Women living with own 
child(ren) 
No 
Yes 
Men who have own child(ren) 
No 
Yes 
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TABLE 5.9 


IMPACTS ON EARNINGS THROUGH MONTH 48, 
BY SELECTED CHARACTERISTICS AT THE TIME OF RANDOM ASSIGNMENT 


(S) 


12,435 
22,290 


23,309 
14,410 
18,434 
15,922 


17,514 
10,332 
13,528 
13,404 


30,483 
18,930 
24,183 
19,560 


12,596 
11,531 


22,470 
24,436 


Total Earnings, Months 1-48 
Experimentals Controls 


(S) 


11,781 
22,588 


21,243 
14,250 
18,182 
20,663 


16,798 
10,487 
12,107 
13,595 


26,364 
18,577 
25,358 
27,625 


11,666 
11,165 


22,488 
26,671 


Subgroup 
Impact 
($) 


Subgroup Impact 


g 
= 
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Characteristic and Subgroup 
Employed within past year 
No 
Yes 


Prior employment, by gender 
Women employed within 
past year 
No 
Yes 
Men employed within past year 
No 
Yes 


Left school in grade 11 or 12 
No 
Yes 


Received occupational training 
within past year 

No 

Yes 


Age 
16-19 
20 or 21 


Age, by gender 
Women 
16-19 
20 or 21 
Men 
16-19 
20 or 21 


241 


TABLE 5.9 (continued) 


Total Eamings, Months 1-48 
Sample sia ea Controls 


Size ($) 
914 14,267 14,493 
1,027 19,453 18,846 
583 10,387 9,652 
458 14,163 13,554 
331 18,271 20,164 
569 25,436 24,729 
1,140 15,568 15,638 
801 19,054 18,435 
1,615 17,051 16,852 
326 16,790 16,538 
1,425 17,381 16,961 
516 15,990 16,340 
763 13,071 11,917 
278 10,401 11,153 
662 22,347 22,786 
238 22,362 22,310 


Subgroup 
Impact 
(S) 


-226 
607 


0. 670 
0.261 


0.793 
0.881 


0.604 
0.794 


0.300 
0.679 


0.712 
0.979 
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Characteristic and Subgroup 


Marital status 
Ever married 
Never married 


Living in own household or 
with boy/girlfriend 

No 

Yes 


Own AFDC case or receiving 
General Assistance 

No 

Yes 


Own AFDC case 
No 
Yes 


Receiving Food Stamps 
No 
Yes 


Arrested since age 16 
No 
Yes 


Arrested since age 16, 


Size 
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TABLE 5.9 (continued) 


otal 


Experimentais 
(S) 


20,721 
16,615 


16,615 
18,752 


17,544 
15,564 


17,401 
15,589 


16,878 
17,227 


17,089 
16,556 


12,537 
12,566 


22,262 
21,897 


4 
Controls 
(S) 


16,970 
16,769 


16,367 
18,593 


17,351 
15,292 


17,182 
15,394 


16,484 
17,313 


17,024 
15,500 


11,800 
12,998 


23,183 
20,344 


0.746 


0.811 


0.931 
0.557 


0.447 
0.917 


0.437 
0.436 


(continued) 
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Total Eamings, Months 1-48 Subgroup Subgroup Impact 
Sample Experimentais Controls Impact 

Characteristic and Subgroup Size (S) (S) (S) p* ($) p* 

Lived with both parents at 

age 14 -37 0.980 
No 1,264 16,536 16,335 201 0.814 “-- o 
Yes 677 17,896 17,658 238 0.839 *- on 

Reason for leaving regular 

high school “ 0.455 
School-related 797 16,170 14,920 1,250 0.208 o o 
Job-related 161 22,022 22,840 -818 0.706 o-- one 
Other 699 16,877 17,249 372 0.724 oes on 


SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 


NOTES: Calculations for this table used data for all sample members for whom there were 48 months of follow-up survey data, 
including those with values of zero for outcomes and those who were assigned to JOBSTART but did not participate. 

Average experimental and control group outcomes reported here are adjusted means from two-way analysis of covariance 
procedures controlling for up to 31 kinds of difference in characteristics, other than the characteristic used to define subgroups, before 
random assignment. The two categories used as factors were research assignment and, one at a time, the baseline characteristics indicated 
(see Ostle, 1975, p. 454). There may be slight discrepancies in reported sums and differences of these adjusted means because of rounding. 

*A two-tailed t-test was applied to each within-subgroup impact and also to each difference between subgroup impacts. For each 
characteristic with more than two subgroups, an F-test was applied to the interaction between that characteristic and experimental or contro! 
status. The columns labeled ‘p’ are the statistical significance levels of each impact and each difference in impacts or F-statistic: that is, p is 
the probability that sample estimates are non-zero only because of random error. Statistical significance levels are indicated as *** = 1 
percent; ** = 5 percent; * = 10 percent. 

or each characteristic that has only two subgroups, the subgroup impact difference is the impact within the first subgroup, less the 
impact within the second subgroup. 
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TABLE 5.10 


IMPACTS ON YEAR-FOUR EARNINGS, 
BY SELECTED CHARACTERISTICS AT THE TIME OF RANDOM ASSIGNMENT 


Total Months 37-48 § Subgroup Subgroup Impact 
Sample §_ Experimentals = Controls Impact Difference’ 
Characteristic and Subgroup Size ($) ($) ($) p* ($) 
Gender -124 
Women 1,041 4,081 3,728 353 0.333 soe 
Men 900 7,342 6,865 477 0.223 oe 
Ethnicity is 
White, non-Hispanic 172 6,644 6,372 272 0.762 “ 
Black, non-Hispanic 860 4,522 4,523 “1 0.998 “ 
Hispanic 847 6,400 §,518 882°* 0.030 o- 
Other 62 6,393 6,361 32 0.983 “ 
Ethnicity, by gender “-- 
‘ Women 
=~ White, non-Hispanic 97 4,730 §,327 -§97 0.617 o-- 
° Black, non-Hispanic 467 3,265 3,387 -122 0.823 -- 
Hispanic 451 4,716 3,718 998° 0.073 o- 
Other 26 5,382 4,712 670 0.775 o- 
Men 
White, non-Hispanic 75 9,071 7,618 1,453 0.295 o- 
Black, non-Hispanic 393 5,910 §,622 88 0.882 - 
Hispanic 396 8,373 7,656 717 0.226 o- 
Other 36 7,662 8,158 -496 0.805 - 
Parental status — 
Women living with own 
child(ren) 
No 533 4,105 3,575 530 0.299 see 
Yes 508 3,928 3,759 169 0.747 s+ 
Men who have own child(ren) 
No 785 7,343 6,752 591 0.161 = 
Yes 115 7,947 8,260 -313 0.778 oe 
247 


=)9 1" 


Characteristic and Subgroup 
Employed within past year 
No 
Yes 
Prior employment, by gender 
Women employed within 
past year 
No 


Yes 
Men employed within past year 
No 


Yes 
Left school in grade 11 or 12 
No 
Yes 


Received occupational training 
within past year 
No 


Yes 
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Sample 


Size 


914 
1,027 


TABLE 5.10 (continued) 


($) 


4,589 
6,487 


3,232 
4,854 


6,087 
8,338 


5,250 
6,084 


5,668 
5,187 


5,856 
4,868 


4,350 
3,291 


7,604 
6,686 


hs 


Total Eamings, Months 37-48 
Experimentals Controls 


($) 


4,527 
5,765 


3,123 
4,172 


6,097 
7,559 


4,704 
5,867 


§,239 
4,921 


5,176 
5,198 


3,758 
3,587 


6,821 
7,073 


Subgroup 


($) 


62 
722*°° 


J 


() 
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Characteristic and Subgroup 


Marital status 
Ever married 
Never married 


Living in own household or 
with boy/girlfriend 
No 


Yes 


Own AFDC case or receiving 
General Assistance 

No 

Yes 


Own AFDC case 
No 
Yes 


Receiving Food Stamps 
No 
Yes 


Anested since age 16 
No 
Yes 


Arrested since age 16, 
by gender 
Women 
No 
Yes 
Men 
No 
Yes 


Sample 
Size 


TABLE 5.10 (continued) 


Total Earnings, Months 37-48 
Experimentals Controls 
($) ($) 
6,141 5,204 
5,534 5,178 
5,498 §,087 
6,010 5,579 
5,900 5,427 
4,751 4,513 
§,777 5,324 
4,918 4,659 
§,731 §,037 
5,357 5,418 
§,613 §,341 
5,471 4,276 
4,100 3,716 
5,076 4,801 
7,391 7,268 
6,943 §,379 


Subgroup 
Impact 


($) 


0.349 


0.789 
0.043 


=E9I- 


impact 
Characteristic and Subgroup Size (8) ($s) (8) p* 


(S) p* 
Lived with both parents at 
age 14 533 0.342 
No 1,264 5,391 5,166 225 0.498 - o- 
Yes 677 5,979 §,222 757* 0.094 - _ 
Reason for leaving regular 
high school - 0.796 
School-related 797 5,249 4,650 599 0.120 -- oe 
Job-related 161 7,162 6,520 642 0.447 - “ 
Other 699 5,649 5,406 243 0.553 - “ 


SOURCE: MODRC calculations from JOBSTART enroliment form and survey data. 


NOTES: Calculations for this table used data for all sample members for whom there were 48 months of follow-up survey data, 
including those with values of zero for outcomes and those who were assigned to JOBSTART but did not participate. 

Average experimental and control group outcomes reported here are adjusted means from two-way analysis of covariance 
procedures controlling for up to 31 kinds of difference in characteristics, other than the characteristic used to define subgroups, before 
random assignment. The two categories used as factors were research assignment and, one at a time, the baseline characteristics indicated 
(see Ostle, 1975, p. 454). There may be slight discrepancies in reported sums and differences of these adjusted means because of rounding. 

*A two-tailed t-test was applied to each within-subgroup impact and also to each difference between subgroup impacts. For each 
characteristic with more than two subgroups, an F-test was applied to the interaction between that characteristic and experimental or control 
status. The columns labeled ‘p* are the statistical significance levels of each impact and each difference in impacts or F-statistic: that is, p is 
the probability that sanple estimates are non-zero only because of random error. Statistical significance levels are indicated as *** = 1 
percent; ** = § percent; * = 10 percent 

or each characteristic that has only two subgroups, the subgroup impact difference is the impact within the first subgroup, less the 
impact within the second subgroup. 
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among these subgroup impacts also were not significant, as shown by the right-most p-value for that 
panel (0.522).!6 
The subgroups based on parenting sta \s by gender also show stronger impacts for women than 


for men, and for other women than for ¢:: todial mothers, although here again the differences in 
impacts were not statistically significant. Thus, the earlier observed differences in 48-month earnings 
impacts for men, custodial mothers, and other women did not disappear (although they were not 
statistically significant) when other measured characteristics of each subgroup were controlled for 
in the calculations. The observed male-female earnings differences in impacts do not appear to have 
resulted from other measured non-gender differences in the sample members’ pre-random 
assignment characteristics (such as differences in education or work experience). 

Among the other subgroup impacts on earnings presented in Tables 5.9 and 5.10, all but two 
of the subgroup impact differences (to the right of the rule) were insignificant in a statistical sense. 
The exceptions were the final-year earnings impact (Table 5.10) for the two subgroups defined by 
age alone, where the younger subgroup had a larger impact. Four-year earnings impact differences 
between subgroups seem sizable in absolute terms (a more than $800 difference between the highest 
and lowest subgroup impacts within a category) for ethnicity, ethnicity by gender, parenting status, 
prior employment by gender, age by gender, marital status, arrest history, arrest history by gender, 
and reason for leaving regular high school. 

Reviewing the findings presented in this section, there is no clear pattern among the 
differences in four-year earnings impacts for subgroups. In some cases, subgroups with sample 
members who had greater barriers to employment (represented by lower control group earnings) 
had greater impacts. Women had larger impacts than men, and those arrested between age 16 and 
random assignment had larger impacts than those who had not been arrested. For other subgroups, 
however, the opposite pattern emerged. Impacts were higher, for example, for those with recent 
work experience. For some subgroups, most notably those based on ethnicity, no clear pattern 
emerged as to how impacts were related to employability. For example, for the entire four-year 
period, white, non-Hispanic men and Hispanic women had the highest earnings impacts, and men 
in the “other” subgroup (primarily Asians and Native Americans) had large negative impacts. 


164s mentioned in Chapter 4, no single subgroup impact difference can be calculated for characteristics 
such as ethnicity that have more than two subgroups; consequently, for those characteristics the subgroup 
impact difference column has no number. 
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These mixed findings differ from the results of several studies of employment programs for 


welfare recipients that did find patterns among subgroup impacts.!” In these programs, impacts 
tended to be small or nonexistent for the most and least job-ready, and positive for those who fell 
between these two groups. The subgroups used in the JOBSTART analysis were much more 
narrowly defined, which may have contributed to the absence of a pattern in the impacts. 


Earlier sections of this chapter provided estimates of JOBSTART's impacts on earnings for the 
full sample and key demographic subgroups. This final section addresses another important issue: 
Were these impacts affected by program characteristics? ‘The most direct way to explore this issue 
within the context of the JOBSTART Demonstration is to examine differences in individual site 
impacts, since there were some programmatic differences among the 13 sites and the programs 
operated in a variety of settings. 

This analysis of earnings impacts across sites, of necessity, is less certain than that for the full 
sample and subgroups for two major reasons. First, the JOBSTART sites vary on many dimensions, 
so it is difficult to isolate the influence of any single factor on site-level impacts.'® The various 
ways in which the sites differed are briefly summarized below. Second, the impacts on earnings at 
the individual site level (even when noticeably different from zero) were usually not statistically 
significant. This is the case because the sample size in each site was relatively small; a site’s program 
did not affect the behavior of all youths in the same way and controls’ earnings also varied (in 
technical terms, these last two points suggest a large variance in outcomes within each site); and 
many of the reported earnings impacts were not large. Furthermore, the differences in the individual 
site impacts on employment and earnings were also not statistically significant, suggesting again that 
lessons on program structure can only be tentative.!9 


17Gueron and Pauly, 1991. 

18To answer the question of how program characteristics affected impacts with the same methodological 
rigor as that of the previous analysis of earnings impacts, youths in individual sites would have had to be 
randomly assigned to one of the several types of programs of interest (for example, one emphasizing education 
or one emphasizing training), and individual sites would have had to offer more than one type of program. 
This kind of study — known as a “differential impact" research design — is needed to separate the influence 
of program type from other site variations such as local labor market conditions and differences in the kinds 
of youths drawn to the sites. : 

19ip this context, statistical significance indicates whether the extent of variation in impacts across sites 
was so systematic that it was unlikely to have arisen by chance. 
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Nevertheless, the JOBSTART Demonstration did provide an opportunity to learn more about 


the link between program characteristics and impacts. For example, if site categories based on 
specific characteristics such as program structure or sponsoring organization had shown clear 
differences in impacts, certain lessons might have been drawn (including the lack of influence of 
other factors). 

As the findings in this section show, however, no clear pattern emerged: that is, no one site 
category did appreciably better than the others in terms of four-year impacts. Instead, both positive 
and negative impacts appeared within each category, suggesting that JOBSTART can produce 
positive or negative impacts in a variety of settings and with a variety of program structures. 

Much of the previous analysis in this report divided sites into three groups based on program 
structure — concurrent, sequential/in-house, and sequential/orokered — because these categories 
helped explain differences in program implementation and participation among the sites.” This 
chapter preserves these site groupings, since the implementation research led to a hypothesis that 
these differences in program structure could influence impacts. But it is important to keep in mind 
that these categories are not a neat division of the sites into homogeneous groups.2!_ Thus, 
disentangling the independent effect of any single program feature on impacts across sites can be 
very difficult. 

Despite these difficulties in analysis and interpretation, this chapter does try to draw some 
conclusions about the link between program features and impacts. It attempts this type of analysis 
because of the strong interest in identifying more effective ways to structure education and training 
programs.” The framework for analyzing site-level impacts is presented below, followed by a 
discussion of site-level impacts for the full 48-month sample. 


A. A Framework for Anulyzing Site Variation in Impacts 


The differences in program structure among the sites (concurrent, sequential/brokered, and 
sequential/in-house) were among the most obvious possible influences on site impacts. However, 


For example, the sequential/orokered sites all had difficulty getting young people to make the transition 
to training, and the concurrent sites all tended to emphasize training (as opposed to education) more than the 
other sites. 

210ne key difference within the concurrent category was discussed earlier in the report: Two sites 
(CET/San Jose and Chicago Commons) integrated education into the training curriculum to a much greater 
extent than the other concurrent sites. Other characteristics that varied within categories included the extent 
to which applicants were initially screened, the quality of sites’ implementation of the four JOBSTART 
com ts, and local labor market conditions. 

interest was intensified by the recent publication of the impact report on the Minority Female 
Single Parent (MFSP) Demonstration, sponsored by The Rockefeller Foundation. See Burghardt et al., 1992. 
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these program factors are entangled with many other sources of variation across sites, including the 


target group recruited at various sites; sites’ greater interest in education or training; the extent of 
screening by site operators; the point at which random assignment took place for the evaluation; the 
availability of alternative local services for members of the control group; and environmental factors 
such as the wage structure and tightness of the labor market. 

1. Program structure. As detailed in Chapter 3, dimensions of program structure that are 
of special interest include concurrent versus sequential education and training classes; the extent of 
integration of education and training; brokering of services among multiple vendors versus in-house 
provision of all services; months of program activities offered and delivered; the relative emphasis 
on education as opposed to occupational skills training; the strength of job placement efforts; and 
the intensity of support services. Table 5.11 (which is the same as Table 3.11) highlights key aspects 
of the sites’ program structure. Sites are grouped under the headings concurrent, sequential/ 
in-house, and sequential/brokered. 

“Average total hours" is one measure of the intensity of program services in a site and "average 
length of stay" is an important dimension of site variation because it can influence how long 
participants lose initial earnings because they are active in the program.2* “Average hours per 
month" is a measure of the extent to which education, training, and other activities were 
concentrated or spread out over time, In general, the shorter the length of stay, the higher were 
the hours per month. CET/San Jose and Chicago Commons, the two sites with the shortest length 
of stay, had high average hours per month because they squeezed just about as much program 
activity as the average site offered into shorter periods of time.”4 

The “average hours of education" column shows much variation, but there was a tendency for 
the sequential — and especially sequential/orokered — sites to emphasize education more than the 


231n sites where experimentals participated for shorter periods of time, there was more post-program time 
during which experimental employment outco:nes could have overtaken control outcomes during the four-year 
follow-up period. CET/San Jose and Chicago Commons were notable on this score, with average lengths of 
stay less than two-thirds of the full-sample average, even though total hours were higher than average at 
Chicago Commons and 84 percent of the average at CET/San Jose, suggesting that shorter lengths of stay are 
not necessarily correlated to lower intensity of program services. 

/San Jose and Chicago Commons had two things in common that help explain their relatively short 
length of stay. First, the programs were concurrent, offering education and training at the same time rather 
than one after the other. Second, there was less emphasis on education leading to a GED than on skills 
training in these integrated or partly integrated programs. Thus, youths in these programs typically stayed 
fewer months than those in sequential programs and in other concurrent programs that emphasized education 
more. For these reasons, one might hypothesize that earnings losses would have been confined to a shorter 
period in these two sites than in sites that provided the same level of services over a longer period of time. 
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Average JOBSTART 
Average = Length Average __Average Hours in__——_ Level of Overall Operating 
Total of Stay Hours Initial Job Support Rating of Costs per 
Ske Hours (Months) per Month Education Training Screening Placement Experimental ($)" 
Concurrent 387 61 63 101 276 ee sa asi iat es 
Atlanta Job Corps 334 5.8 50 102 177 High Low High Low 4,100° 
CET/San Jose” 335 at 62 269 309 Low High Medium High 2,000 
Chicago Commons® 422 43 96 69? 353 High High Medium High 6,400 
(Pittsburgh) 538 95 87 99 439 + « Medum Medium High High 5,200 
East LA Skits Center 370 57 65 76 294 = Medium Low Medium Medium 4,900 
EGOS (Derwer) 270 71 38 128 142 Low Low Medium Low 2,000? 
Phoenix Job Corps 420 63 67 161 202 High High High High 4,700° 
SEFYCorpus Chvist! 400 5.0 80 118 262 Medium Low High Medium 2,100 
in-house 518 7.0 75 146 276 ns wi a on Pe 
El Centro (Datias) 406 60 68 147 179 Medium Medium Medium High 5,100 
1 LA Job Corps 607 78 79 146 355 High ss Medium High High 5, 
toa) 
eo 
'  Sequentialibrokered 307 7.7 39 176 71 a ions ia set an 
Allentown (Buffalo) 427 10.7 38 238 105 Medium Low High Medium 5,900 
BSA 262 5.6 49 149 63 Low Low Medium Low 7,500 
CREC (Hantord) 171 63 29 124 35 Low Low Medium Low 5,200 
All sites 396 66 60 125 238 os ‘ss ef Si a5 


SOURCES: MDRC calculations from JOBSTART enroliment form and MIS data (participation figures); MDRC operations staff (implementation ratings); Appendix C (costs). 


NOTES: Calculations for this table used data for all 968 experimentais for whom there were 48 months of follow-up survey data, including those with values of zero for 
outcomes end those who were assigned to JOBSTART but did not participate. 

®these cost estimates are rounded to the nearest $100. Al costs are in 1986 dollars. 

Othe estimates do not include the cost of providing on-ste medical and dental services. The value of these services per JOBSTART experimental was approximately $400 in 
the Atianta ste, $400 in Phoenix, $600 in Los Angeles, and $24 in Denver. 

Cin this site, education end training were more integrated than in other sites, and staff strongly emphasized training over pessing the GED examination. 

in this site, some education hours ere included in the training component hours. 
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concurrent sites.> The “average hours in training" column shows even more site variation than 


the previous columns. As already indicated, the dramatic difference between the sequential/ 
brokered sites and the others on this score probably reflected both the systemic and administrative 
problems the former encountered in linking education and training, and differences in the interests 
and expectations of recruits.2° One surprise here is that the two sequential/in-house programs 
managed to deliver hours of training comparable to those in concurrent programs.2” The rating 
of job placement is a subjective ranking of the strength of job placement assistance efforts in a site, 
and the rating of support services assesses the strength of the implementation of that component 
of JOBSTART.4 

To sum up Table 5.11, CET/San Jose, Chicago Commons, Connelley in Pittsburgh, El Centro 
in Dallas, and the Los Angeles and Phoenix Job Corps programs seem to have implemented 
JOBSTART'’s four components more fully than did the other sites. Thus, these sites appear to have 
set the stage for stronger program impacts than those in the other sites. However, the relatively 
strong treatments, short lengths of stay, and resulting intensity (hours per month) at CET/San Jose 
and Chicago Commons could have given these two sites an edge over the others by lessening initial 
earnings losses during the period of program participation. 

2. Differences in applicant characteristics. While all JOBSTART sites had a common 
target group for the demonstration (disadvantaged young dropouts reading below the eighth-grade 
level), there were variations in site recruiting emphases and in participant characteristics. These site 
variations arose because different types of youths applied and because intake practices were not 
uniform. 

Some characteristics of youths varied in easily measured ways. For example, as reported in 
Chapter 2, sites varied greatly in the proportion of the sample that was made up of youths who had 
recent work experience or who were custodial mothers. Factors such as these were measured at 
random assignment and can, to a considerable extent, be adjusted for statistically in comparing 
program impacts across sites. One type of impact estimate reported later in this section does include 


Three of the eight concurrent sites averaged less than two-thirds of the average for all sites, and only 
one of them substantially exceeded the all-site average. Variation in the proportion of youths receiving any 
education was not a major factor in site variation in hours, except at CET/San Jose (which did not offer 
separate education classes), where the proportion varied in a narrow range around the full-sample average. 

one-quarter of experimentals at sequential/orokered sites got any occupational skills training at all. 

27This achievement is notable because only half of the experimentals in sequential/in-house sites stayed 

in the program long enough to receive any training, while more than 80 percent of experimentals in concurrent 
sites received training. 

Costs, in the right-hand column, are discussed in more detail in Chapter 7. 
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such statistical adjustments in an effort to control for these measured differences in sample 
characteristics among sites and to move closer to isolating variation in impacts caused by program 
structure.2? 
Some sample characteristics are much harder to observe and therefore were not measured at 
entry into the sample; these cannot be included .n statistical adjustments. One especially important 
factor is the youths’ goals for participating in the program and their resulting interest in the various 
JOBSTART components. Although the youths’ goals and interests were not observed directly in 
any quantifiable way, the reputation of the site in the community and its perceived service emphasis 
probably had a strong influence on who applied for the program.” 
3. Screening. Some sites — such as CET/San Jose, EGOS in Denver, BSA in New York 
City, and CREC in Hartford — did not screen applicants much before they entered the sample, 
while others (such as Chicago Commons and the Job Corps sites) carried out extensive 


screening”! When a great deal of screening takes place, it is reasonable to assume that those 
randomly assigned (including those assigned to control group status) will have high levels of 
perseverance, motivation to get into a program, motivation to work, and other important 
characteristics hard to measure directly. Thus, all else being equal, in sites that screened heavily, 
control group outcomes should have constituted tougher benchmarks for experimentals to surpass 


2%although these impacts include linear statistical adjustments for sample characteristics at random 
assignment, not all relevant characteristics were measured; there might be differential errors of measurement 
of characteristics; true relationships between impacts and characteristics may be nonlinear; or impacts for sites 
with very unusual sample characteristics might be difficult to adjust for properly with any statistical model. 
Thus, adjusted site outcomes and impacts must be viewed with a special caution not applicable to estimated 
sul p impacts. 
sites with a history of providing basic skills instruction and not occupational training (BSA in New 
York City, Allentown in Buffalo, and CREC in Hartford) naturally would seem to have attracted youths who 
were more interested in GED attainment than in immediate acquisition of marketable occupational skills, 
while those sites emphasizing occupational skills training (CET/San Jose and Chicago Commons) seem to have 
attracted youths who were more interested in job skills and immediate employment than in education. 
Available information does indirectly support this generalization. Table 5.11 shows that JOBSTART hours 
of education tended to be high and that hours of occupational skills training were lowest in 
sequential/orokered sites, although the latter clearly resulted partly from the administrative problems those 
sites encountered in linking education and training. Taken together, these findings are consistent with the idea 
that recruits in the sequential/brokered sites were less interested in occupational training per se than in 
education leading to a GED. In other words, these sites may have delivered fewer hours of skills training 
partly because their recruits demanded fewer hours. 

Chapter 3 pointed out the severe recruitment pressures at CET/San Jose during intake for JOBSTART. 
CET/San Jose may have been able to adopt this "no screening" policy because of the characteristics of 
disadvantaged youths in its service area. Even with open admissions, a higher percentage of the CET/San Jose 
youths had recent work experience than did the full sample. This probably occurred because CET/San Jose 
drew largely Hispanic youths, a group that in the western states typically has a higher labor force participation 
rate than other minority school dropouts. 
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than in sites that carried out minimal screening. However, minimal screening may yield groups of 


experimentals who are harder to keep in the program and harder to place in jobs at completion. 

4. Point of random assignment. In addition to the normal steps and resulting screening 
in recruitment and intake, the research design introduced random assignment, which could occur in 
different sites at slightly different points relative to initial contact with a potential sample member 
and actual start-up of services. As mentioned in Chapter 2, putting random assignment early in the 
program flow (as at CET/San Jose) tended to include in the sample individuals who might drop out 
during the later steps of intake, while putting random assignment relatively late (1s at Allentown in 
Buffalo) meant that only those making it to that stage of intake were in the research sample. Late 
random assignment tended to raise the proportion of the experimental group participating in 
JOBSTART, because it meant that those randomly assigned were interested enough in the program 
to persevere through the steps of intake and because it reduced the waiting period between random 
assignment and the start of services. Late random assignment also tended to raise the proportion 
of controls who were served in alternative programs, because those randomly assigned to the control 
group were also quite motivated to receive services.°2 

5. Services received by controls. Since the impacts reported here are the effects of 
JOBSTART compared to the alternative services in which the controls participated (rather than 
those of JOBSTART versus no services), variation in the controls’ benchmark levels of services will 
influence impacts. As discussed in Chapter 3, in most sites more than 90 percent of experimentals 
were active in education or training at some point in the four-year follow-up period. There was one 
notable exception: At CET/San Jose, a lower than average percentage of experimentals 
(approximately 75 percent) participated during the four years because of program funding problems 
early in random assignment. In contrast to the consistent and high level of participation among 
experimentals, the level of control services varied greatly by site. Over the four years, the 
percentage of controls active in education or training ranged from a high of 78 percent at the 
Atlanta Job Corps to a low of 40 percent at El Centro in Dallas. Consequently, the service receipt 
differences between experimentals and controls also varied greatly by site, from a high of 60.5 
percentage points at El Centro to a low of 19.2 percentage points at the Atlanta Job Corps.* 


32The correlation coefficient of sites’ percentage of experimentals and controls participating in education 
and training is +.37. The point of random assignment thus had implications for the proportion of 
tals and controls receiving services, but both groups were affected similarly, so there was no 
consistent effect on the difference in the proportion of experimentals and controls receiving some type of 
education or training. 
33See Cave and Doolittle, 1991, for the details of these differences in service receipt across sites. 


-171- 


263 


Because of the low percentage of experimentals active in JOBSTART at CET/San Jose, the site had 
a service receipt differential of 27 percentage points. 

6. Wage structure and labor market tightness. In some communities, jobs tend to be 
plentiful but low-paying; in others, they tend to be scarcer but higher-paying; and in still others, they 
are both scarce and low-paying. This could be the result of long-term differences in the industrial 
base of the communities or of how they are affected by business cycles. These labor market 
conditions could affect both the control group’s level of employment and earnings and the 
availability of jobs that reward the experimental group’s increased educational attainment and 
occupational training. 


As just discussed, sites differed in many ways, making it very difficult to isolate the influence 
of program features on impacts. The most straightforward method of estimating impacts is to 
calculate separate ("split sample") experimental-control comparisons for each site, in effect creating 
13 separate samples (or data files) and calculating impacts for each one. In doing this type of 
impact analysis for sites, the only adjustment made is to take account of differences in the observed 
pre-random assignment characteristics of the site’s experimental and control groups, which may occur 
when sample sizes are relatively small. 

It is possible to move beyond this type of analysis because one source of site variation — the 
observed characteristics of the youths in the sample — can be adjusted for statistically by relying on 
the fact that youths with particular characteristics are rarely concentrated in just one site. This type 
of adjustment takes account of the program impacts for each type of youth in the full 13-site sample 
(for example, the impacts for blacks, those with prior work experience, or those receiving public 
assistance) and adjusts individual site impacts to reflect the fact that some sites served many youths 
who (in the full sample) tended to have lower than average impacts, while other sites served more 
than the usual percentage of youths who had higher than average impacts.>4 

The two analytical approaches answer different questions and each has strengths and 
weaknesses. The split-sample approach is most valuable as a description of what actually occurred 
in each site: that is, it presents the best estimates for the impacts of the site’s program as it actually 


>The statistical adjustment assumes a linear relationship between each characteristic and outcomes, and 
is the same assumption underlying the method in Tables 4.4, 4.7, 5.9, and 5.10, which present impacts for 
designated subgroups controlling for differences between the subgroups other than the characteristic used to 
define them. These tables used linear adjustments of outcomes, with differences in youths’ characteristics 
entered as covariates. See Ostle, 1975. 
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operated for the people recruited and randomly assigned to the experimental group. Its chief 


drawback is that the differences in outcomes for experimentals and controls within each site are not 
fully comparable across sites because the characteristics of the two groups differ from site to site. 
If, for example, youths who were employed in the year prior to random assignment tended to have 
lower employment and earnings impacts, sites with a sample made up of youths with higher than 
average prior-year employment will have lower impacts than if they had served more disadvantaged 
recruits. Furthermore, the individual sites may have relatively small sample sizes, limiting the 
precision of impact estimates. 

The second approach, adjusting for measured site differences in pre-random assignment 
characteristics, is most valuable in trying to isolate the effects of location per se. Sites with 
populations different from the overall average are not penalized or rewarded relative to other sites, 
as they are in the split-sample approach. For example, the estimated impacts for a site with a higher 
than average proportion of youths with prior employment would be larger using this adjustment than 
using the split-sample approach. However, in order to make the adjustment for site differences in 
the mix of youths’ pre-random assignment characteristics, it is necessary to assume that sites could 
have served — and achieved “average” impacts for — populations they actually did not serve. For 
example, sites set up to serve a particular type of participant (for example, young men) might not 
be able to serve other young people. Thus, they might not attain the impact observed in the full 
sample for these other groups.*© 

The outline of the site impact story is the same using both types of approaches, although the 
adjustment does make a noticeable difference in the impacts of a few sites that served a group of 
young people that differed markedly from the sample as a whole and/or had smaller than average 
sample sizes. Table 5.12 presents the split-sample impacts, that is, those not adjusted for differences 


35This was the approach taken, for example, in the recent report on the Minority Female Single Parent 
(MFSP) Demonstration; the findings for each of the four sites were calculated separately. See Burghardt et 
al., 1992. 

The adjusted impact approach uses the entire sample, lessening the problems of sample size in the 
analysis. However, it introduces a further complication: The adjustments implicitly assume that youths’ 
characteristics affect outcomes in the same way in each site. More technically, the assumption is that the 
coefficients estimated for the regression used in adjusting for differences in characteristics (both between 
experimentals and controls and across sites) are the same across all sites. The split-sample approach estimates 
separate regressions for each site to adjust for differences in sample characteristics between experimentals and 
controls. Therefore, the shift from split-sample impacts to those adjusted for site differences in pre-random 
assignment characteristics involves two changes: changing to the standard full sample regression coefficients 
and adjusting for differences across sites in sample characteristics. Consequently, it is often not possible to 
give a simple, intuitive explanation of why the shift from split-sample to adjusted impacts caused the observed 
change in impacts. 
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TABLE 5.12 


IMPACTS ON EARNINGS, BY SITE 
(NOT ADJUSTED FOR SITE DIFFERENCES IN SAMPLE CHARACTERISTICS 
AT THE TIME OF RANDOM ASSIGNMENT) 


Total Earnings, Months 25-48 


Sample Experiments Controls Difference Experimentals Controls Difference 

Ste Size (3) (S) (S) (S) (S) 
Concurrent 

Atlama Job Corps 69 10,492 8,241 2,251 15,123 16,498 -1,375 

CET/San Jose 167 20,808 14,721 6,067*** 32,959 26,244 6,715** 

Commons 75 9,015 12,471 -3,456 16,346 19,172 -2,826 

Conneliey (Pittsburgh) 184 6,188 §,148 1,040 9,047 8,125 922 

East LA Skills Center 106 15,857 14,542 1,315 23,705 23,586 119 

EGOS (Denver) 198 10,415 11,441 -1,026 15,361 17,828 -2,447 

Phoenix Job Corps 134 6,835 9,685 -2,850 11,393 17,729 -6,336** 

SER/Corpus Christi 247 11,065 10,401 684 16,075 17,244 -1,169 
Sequential/in-house 

El Centro (Dalias) 179 11,018 9,989 1,029 16,568 14,030 2,538 

LA Job Corps 231 11,710 11,683 27 17,758 18,869 “1,111 
Sequential/brokered 

Allentown (Buffalo) 135 5,556 5,081 475 9,338 8,229 1,109 

BSA (NYC) 117 9,771 10,782 1,011 15,465 21,642 6,177 

CREC (Hartford) 99 8,893 9,897 +1,004 17,302 17,268 34 


SOURCES: MODRC calculations from JOBSTART enroliment form and survey data. 


NOTES: Calculations for this table used data for all sample members for whom there were 48 months of follow-up survey data, including those with values of 
zero for outcomes and those who were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For some sample members, the month of random assignment may be less than a month, 
beginning with the date of random assignment and ending on the last day of the month. 

Average experimental and contro! group outcomes reported here are adjusted means from spiit-file linear analysis of covariance procedures controlling for 19 kinds 
ak tier erenoe in charecietteice behors cenidonn eanignavers teen Ole, 1975, p. 461; Cave, 1967; and Appendix 8). There may be slight discrepancies in reported sums 
and differences of these adjusted means because of rounding. 

A two-tailed t-test was applied to each difference between average experimental and contro! group outcomes. Statistical significance levels are indicated as 
eee = 1 percent; ** = § percent; * = 10 percent. 
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across sites among the youths served. Table 5.13 presents impacts that have been adjusted for these 
differences. Both tables present impacts per experimental and include two impacts: The left panel 
presents experimental-control differences in earnings during the third and fourth years of follow-up 
and the right panel presents differences in earnings over the entire four years of follow-up. 

As anticipated, few of the individual site impacts were statistically significant, so any lessons 
would come from patterns of impacts across sites. In both tables, in each site grouping, there are 
one or more sites with a positive impact and one or more with a negative or very small positive 
impact for each of the two time periods. As the tables show, there was variation in impacts within 
each group of sites as well as between groups of sites. Impacts on third- and fourth-year earnings 
showed great apparent variation across sites (though the differences were not statistically significant) 
and considerable apparent variation within each of the three groups of sites. Because of the diverse 
earnings of youths within the individual sites and the small samples (owing to the fine level of 
disaggregation), only one of the 13 individual site impacts for years three and four was statistically 
significant in each table, and for the entire four-year follow-up period, Table 5.12 shows only two 
significant impacts and Table 5.13 only one. It is notable that there were positive and negative 
earnings impact estimates in each of the three groups of sites in both tables. 

The most striking finding in these tables is the strong impacts for CET/San Jose. In both 
tables, and for both the last two years and the entire four-year follow-up period, earnings impacts 
were large, positive, and statistically significant. These occurred despite CET/San Jose’s lower than 
average experimental-control service receipt differential, discussed above. Further, because of 
experimentals’ lower than average participation rate at CET/San Jose, its impacts per JOBSTART 


participant would be substantially higher than at other sites.*” 


These findings, coupled with 
CET/San Jose's large positive impacts in the Minority Female Single Parent study (see Burghardt 
et al., 1992), are mounting evidence of the effectiveness of this program. 

That said, it is very difficult to identify what features of the CET/San Jose approach led to its 
strong impacts. Site characteristics include a relatively concentrated dose of JOBSTART (i.c., a 
short average length of stay but near-average total hours), strong job placement efforts, little up- 
front screening of applicants, close ties to the employer community, integration of education and 


training, a long-standing good reputation in the local communities, a clear organizational mission, 


3"Impacts per JOBSTART participant can be calculated by dividing impacts per experimental by their 
participation rate. 
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TABLE 5.13 


IMPACTS ON EARNINGS, BY SITE 
(ADJUSTED FOR SITE DIFFERENCES IN SAMPLE CHARACTERISTICS 
AT THE TIME OF RANDOM ASSIGNMENT) 


Total Eamings, Months 25-48 Total Earnings, Months 1-48 
Sample Experimentais Controls Difference Experimemais Controls Difference 
Site Size (S) ($) (S) (8) ($) (S) 
Concurrent 
Atlanta Job Corps 69 12,205 10,112 2,093 18,447 19,234 -787 
CET/San Jose 167 18,909 12,362 6,547*** 29,600 22,252 7,342*** 
Chicago Commons 75 10,309 11,726 1,417 18,125 18,296 171 
Conneliey (Pittsburgh) 184 7,470 6,685 785 10,935 10,375 560 
East LA Skills Center 106 14,501 13,158 1,343 22,002 21,355 647 
EGOS (Denver) 198 11,091 10,690 401 16,268 16,920 652 
Phoenix Job Corps 134 6,873 8,196 +1,325 11,396 14,509 3,113 
SER/Corpus Christi 247 8,477 7,992 485 11,705 13,327 -1,622 
Sequentiallin-house 
El Centro (Dallas) 179 11,393 11,057 336 17,111 15,506 1,605 
LA Job Corps 231 12,636 12,757 -121 19,570 20,789 -1,219 
Sequential/brokered 
Allentown (Buffalo) 135 7,481 6,577 904 12,589 11,174 1,415 
BSA (NYC) 117 11,923 10,499 1,424 19,682 20,825 1,143 
CREC (Hartford) 99 9,754 11,124 1,370 19,017 19,302 -265 


SOURCES: MODRC calculations from JOBSTART enrolment form and survey data. 


NOTES: Calculations for this table used data for all sample members for whom there were 48 months of follow-up survey data, including those with values of 
zero for outcomes and those who were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For some sample members, the month of random assignment may be less than a 
NEE RE ee ee er eet re eeas han an ekieed ron a ion 

verage experimental and contro! group outcomes reported here are adjusted means from two-way analysis of covariance procedures controlling for 19 kinds of 

disses clues, bie: tan ee. Gales sardomn akmarenert The two categories used as factors were research assignment and site (see Ostle, 1975, p. 
454). There may be slight discrepancies in reported sums and differences of these adjusted means because of rounding. 

A two-tailed t-test was applied to each within-site impact. An F-test was applied to the interaction between site and experimental or control status. Statistical 
significance levels are indicated as *** = 1 percent; ** = 5 percent; * = 10 percent. - 
e() 


and experienced and skilled staff. It also operated in an unusual labor market, as evidenced by the 
high control group earnings throughout the follow-up period.” 

As an example of the difficulty of isolating the influence of individual program features, 
Chicago Commons was the site most similar to CET/San Jose (although it screened applicants more, 
the labor market in Chicago was very different from that in San Jose, and training tended to be 
concentrated in a smaller number of occupations). In the first two years of follow-up, earnings 
impacts adjusted for differences in sample characteristics were positive in the two sites, and impacts 
in the second year (a post-program period for most experimentals) were the largest of any sites. 
However, in the third and fourth years of follow-up, earnings impacts continued to be strongly 
positive in CET/San Jose while turning strongly negative for Chicago Commons.°? 

In addition to CET/San Jose, five other sites had positive earnings impacts for years three and 
four of follow-up in excess of $1,000 in Table 5.12 or 5.13, and their diversity illustrates again the 
difficulty of drawing conclusions. The Atlanta Job Corps, Connelley in Pittsburgh, and East LA 
Skills Center were all concurrent sites, with the latter two based in adult vocational schools. El 
Centro in Dallas, based in a community college, operated a sequential/in-house program and Basic 
Skills Academy in New York City was a community-based organization operating a sequential/ 
brokered program with a heavy emphasis on education. 

Despite strong policy interest in the influence of program structure on impacts, no simple story 
emerges from the data. JOBSTART was sometimes effective and sometimes ineffective in yielding 
labor market gains in brokered programs and in-house programs; it also showed varying success in 
both concurrent and sequential programs. This suggests that the JOBSTART program model can 
be implemented successfully in a variety of settings. The one clear conclusion from this analysis 
is the growing support for the effectiveness of the CET/San Jose program. 


38CET has received a grant from the U.S. Department of Labor to replicate its program in 10 sites. This 
effort includes four sites where CET will initially operate the local programs itself and six sites where CET 
salt s are training other organizations. 
One hypothesis, which cannot be tested with available data, is that Chicago Commons’ focus on 
training for a few occupations in high demand in the local economy initially paid off for 
masher but then the labor market changed and participants were vulnerable. 
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Chapters 4 and 5 presented estimates of JOBSTART's impacts or educational attainment, 
employment, and earnings for the full 48-month sample and key subgroups. If JOBSTART was 
successful, other aspects of the JOBSTART youths’ lives may also have changed. The following 
questions about non-labor market outcomes are addressed in this chapter for the full sample and 
for subgroups among which there are notable differences in impacts, including subgroups defined 
by gender, parenting status, and — for some outcomes — criminal record prior to random 
assignment.! 


* Did JOBSTART reduce receipt of public benefits? 


* Did JOBSTART lead to changes in household composition, for example, by 
reducing the birth rate or increasing the marriage rate? 


* Did JOBSTART reduce criminal arrest rates and drug use? 


* Did JOBSTART lead to more productive activity,” defined as either working or 
further education and training? 


Among the outcomes analyzed in this chapter, impacts for the full sample were rarely large or 
statistically significant. When impacts were large or significant, they tended to be for the first 12 
or 24 months of follow-up, essentially the in-program period and the following year.> Among the 
subgroups, women who were not custodial mothers at random assignment and men who had been 
arrested between age 16 and random assignment had more encouraging impacts on some outcomes. 
More specifically: 


1The samples for the impact analyses in this chapter vary slightly owing to missing data for some outcomes. 
In all cases, the sample includes at least 95 percent of the 48-month impact sample analyzed in the previous 
chapters. 

During the latter three years of follow-up, the employment rates of experimentals and controls were quite 
similar: Thus, the answer to this fourth question depends on whether there were differences in how 
experimentals and controls spent nonworking time. 

3 pattern of encouraging in-program impacts that disappeared in the initial years of post-program follow- 
up was seen in several prior studies of youth programs, including evaluations of Project Redirection for young 
teenage mothers (Polit, Quint, and Riccio, 1988) and the Summer Training and Education Program (STEP) 
for at-risk high school students (Walker, 1992). However, in the case of Project Redirection, analysis of five- 
year follow-up data showed a reappearance of several positive non-labor market impacts. Similar long-term 
follow-up data for STEP were not available. 


¢ For women who were not custodial mothers, there was a consistent pattern of 
substantial — and for some periods significant — impacts on AFDC receipt, and 
small, insignificant impacts on pregnancy and childbirth rates (that is, the rates 
were somewhat lower for experimentals than controls). The pattern of findings 
over the four-year period indicates that JOBSTART appears to have lowered the 
probability that this group of young women would receive AFDC benefits, in effect 
serving as a welfare prevention program. 


¢ For custodial mothers, experimentals had significantly higher rates of pregnancy 
and childbirth than controls, and — related to these findings — a mixed pattern of 
impacts on receipt of public benefits. Because the increase in births occurred 
primarily among custodial mothers who had been married at some time prior to 
random assignment, there was no significant increase in AFDC receipt (the 
program principally serves single parents), but there were some increases in receipt 
of other public benefits.‘ 


¢ For all men in the sample, JOBSTART had little impact on any of the outcomes 
studied in this chapter. In general, public benefits were a much less important 
source of income for men than women and there were small experimental-control 
differences in these outcomes. Further, the arrest rates for experimentals and 
controls over the four years were very similar. 


¢ For men with an arrest between age 16 and random assignment — an especially 
disadvantaged subgroup -- JOBSTART led to a significant decline in the use of 
drugs other than marijuana. 
In determining the effect of JOBSTART on public assistance receipt, pregnancy and childbirth, drug 
use, and criminal arrests, this chapter presents a vivid picture of the lives of JOBSTART sample 


members, a group of disadvantaged young peopie. 


Table 6.1 summarizes the impacts of JOBSTART on receipt of AFDC, Food Stamps, and 
General Assistance for the full sample. For each benefit, the table shows impacts on the percentage 
ever receiving the benefit and un benefit income received for the four-year follow-up period and 
for individual years. 

In general, there were very small differences in the percentage of experimentals and controls 
in the full sample who ever received these benefits, during either the entire follow-up period or 
iudividual years. Over the full 48 montis, virtually the same Percentage of controls and 


‘AFDC was available to two-parent families during the follow-up period in many states, but there were 
work history requirements for the “principul earner" that could make young parents ineligible. Food Stamps 
and some other forms of public assistance do not have such restrictions regarding eligibility. 
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TABLE 6.1 
IMPACTS ON AFDC, FOOD STAMPS, AND GENERAL ASSISTANCE 


THROUGH MONTH 48 
Outcome and Follow-Up Period Experimentals Controls _Difference p* | 
Ever received AFDC (%) 
Years 1-4 35.4 35.1 0.3 0.862 
Ever received AFDC (%) | 
Year 1 20.5 19.3 1.2 0.373 | 
Year 2 26.5 26.5 0.0 1.000 } 
Year 3 23.8 24.5 -0.7 0.677 
Year 4 26.5 27.4 0.9 0.587 
Total AFDC income ($) 
Years 1-4 3,636 3,562 74 0.750 
AFDC income ($) 
Year 1 687 624 63 0.243 
Year 2 840 816 24 0.703 
Year 3 984 987 3 0.972 
Year 4 1,125 1,136 “11 0.897 
Ever received Foud Stamps (%) 
Years 1-4 58.2 60.1 -1.9 0.327 
Ever received Food Stamps (%) 
Year 1 36.9 38.5 -1.6 0.381 
Year 2 44.9 46.4 “1.4 0.446 
37.4 36.5 0.9 0.641 
43.4 43.1 0.3 0.879 
2,508 2,532 -24 0.839 
523 568 45 0.207 
537 579 42 0.228 
658 627 31 0.449 
790 759 31 0.493 
962 916 
(continued) 
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TABLE 6.1 (continued) 


Outcome and Follow-Up Period Experimentals Controls Difference 


Ever received General Assistance (%) 
Years 1-4 11.2 


Ever received General Assistance (%) 


Total General Assistance income ($) 
Years 1-4 


General Assistance income ($) 


SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 


NOT-S: Calculations for this table used data for all sample members for whom there were 
48 months of follow-up survey data on the specific outcomes, including those with values of zero for 
outcomes and those who were assigned to JOBSTART but did not participate. 
did not always take place on the first of the month. For some sample 
may be less than a month, beginning with the date of 
month. 


slight discrepancies in reported sums and differences of these adjusted means because of rounding. 
*The column labeled °p* is the statistical significance level of the difference between 


average experimental and contro! group outcomes. Statistical significance levels are indicated as *** 
= 1 percent; ** = 5 percent; * = 10 percent. 


experimentals ever received AFDC or General Assistance, while slightly more controls ever received 
Food Stamps, but this small difference was not statistically significant. Impacts for individual years 
were similarly small, in no case exceeding two percentage points. 

Impacts on income received from the various benefits were also small and insignificant. In 
calculating these impacts, sample members who did not receive a benefit were counted as receiving 
zero income from that benefit, and were included when averaging the total benefit income received 
by experimentals and controls. For example, calculations of total AFDC income over the four-year 
follow-up period — including those who received and did not receive AFDC — show that 
experimentals averaged $3,636 while controls averaged $3,562, for a $74 impact on this source of 
income. If sample members who did not receive AFDC had been excluded from the calculations, 
the average total AFDC income for all experimentals or controls would have been much higher than 
those presented here. 


A. Public Assistance Impacts for Selected Subgroups 

A story begins to emerge when the sample is split into the three key subgroups: JOBSTART 
impacts on receipt of AFDC, Food Stamps, and General Assistance for women who were not 
custodial mothers (usually referred to as “other women" earlier in the report), custodial mothers, and 
men are presented in Tables 6.2, 6.3, and 6.4. 

1. Women who were not custodial mothers. For this subgroup, Table 6.2 shows that a 
lower percentage of experimentals than controls ever received each type of benefit during the entire 
four-year follow-up period and during many of the individual years, with the impacts on AFDC 
receipt in the later years of follow-up being statistically significant.’ Similarly, experimentals 
received less income from each source — a ‘‘iference of $775 for AFDC, $340 for Food Stamps, 
and $209 for General Assistance — with the impact on each outcome very near the usual levels of 
statistical significance. 

Noteworthy are the relatively large and significant differences in the AFDC receipt rates during 
follow-up years two, three, and four. These impacts (8.0, 9.1, and 8.6 percentage points) amounted 
to percentage reductions in AFDC receipt of 27 percent in both years two and three and 22 percent 
in year four. Impacts on welfare receipt of this magnitude are unusual in evaluations of employment 
and training programs and are encouraging evidence that JOBSTART may have served as a welfare 
prevention program for this subgroup. 


SThe small sample size of this subgroup made it unlikely that even substantial differences would be 
statistically significant under the usual tests. 
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TABLE 6.2 


IMPACTS ON AFOC, FOOD STAMPS, AND GENERAL ASSISTANCE THROUGH MONTH 48, 
FOR WOMEN NOT LIVING WITH THEIR OWN CHILD(REN) AT THE TIME OF RANDOM ASSIGNMENT 


Outcome and Follow-Up Period 


Ever received AFDC (%) 
Years 1-4 


Ever received AFDC (%) 
Year 1 
Year 2 
Year 3 


Ever received Food Stamps (%) 
Year 1 
Year 2 
Vear 3 
Year 4 


Total Food Stamp income (S$) 
Years 1-4 


Expezimentals Controls Difference 


38.0 


12.6 


24.1 
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p* 


0.117 


0.150 
0.159 


0.109 


0.411 
0.238 
0.151 
0.168 


0.102 


0.430 
0.267 
0.182 
0.138 


( 


TABLE 6.2 (continued) 


Outcome and Follow-Up Period Experimentals Controls Difference p* 


Ever received General Assistance (%) 
Years 1-4 10.1 13.5 3.4 0.178 


Year 1 7.0 7.6 -0.5 0.784 

Year 2 6.2 6.4 -0.2 0.912 

Year 3 2.9 5.8 -2.9° 0.098 

Vear 4 3.9 5.9 -2.0 0.277 
Total General Assistance income ($) 

Years 1-4 301 510 -209 0.106 
General Assistance income (S$) 

Vear 1 120 122 -2 0.953 

Year 2 65 66 -1 0.959 

Year 3 56 154 -98* 0.052 

Year 4 60 167 -107* 0.053 
Sample size 279 242 


SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 


NOTES: Calculations for this table used data for all sample members for whom there were 
48 months of follow-up survey data on the specific outcomes, including those with values of zero for 
outcomes and those who were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For some sample 
members, the month of random assignment may be less than a month, beginning with the date of 
random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted means from 
linear analysis of covariance procedures controlling for up to 29 kinds of difference in characteristics 


average experimental 
= 1 percent; ** = § percent; * = 10 percent. 


The levels of receipt of benefits for individual years reveal the changing circumstances of these 


young women’s lives. In the initial year of follow-up, 12.6 percent of experimentals and 14.9 percent 
of controls received AFDC; 33.6 percent of experimentals and 36.5 percent of controls received 
Food Stamps; and slightly more than 7 percent of both groups received General Assistance. In the 
three succeeding years, increasing percentages of both groups received AFDC and Food Stamps, 
but the increase in receipt was smaller for the experimental group, leading to differences in the 
percentage receiving AFDC (for example, 39.3 percent for controls versus 30.7 for experimentals 
in year four) and — to a lesser extent — Food Stamps and General Assistance. The impacts on 
benefit income received during the individual years reflect the same pattern, with larger welfare 
savings in the two later years. On a monthly basis, the pattern is also present, with fewer 
experimentals than controls receiving AFDC in most months (not shown in the table). 

2. Custodial mothers. Because of their parenting status at random assignment, custodial 
mothers were more likely to receive public benefits than other women in the sample. For example, 
among custodia! mothers more than 60 percent of experimentals received AFDC and Food Stamps 
in the first year of follow-up and more than 80 percent received them at some point within the four- 
year period (see Table 6.3). Except for impacts on the percentage receiving Food Stamps in years 
two and three, none of the impacts on the percentage receiving a specific benefit was either large 
or statistically significant. In the case of Food Stamps, there was also an abrupt change in the 
direction of impacts between years two and three: 6.1 percent fewer experimentals than controls 
received Food Stamps in year two, but by the following year 7 percent more received them. In 
general, there is not any clear trend over time in these impacts. 

Over the entire four years of follow-up, Table 6.3 indicates that there was virtually no 
difference in AFDC payments to custodial mothers between experimentals and controls, although 
the annual differences that emerged in years three and four are somewhat encouraging. Over the 
four years, the experimentals received an average of $112 more than the controls in Food Stamps 
($4,872 versus $4,760) and $359 more for General Assistance ($744 versus $385). The experimental- 
control differential was statistically significant for General Assistance, but not for Food Stamps. The 
statistically significant experimental-control differences in General Assistance payments in years 
three and foar ($123 and $164, respectively) were the primary reason for the large difference over 
the 48 months. 

3. Men. The public benefits analyzed in this section were much less important for men 
than women, with about 10 percent or less of the men ever receiving AFDC or General Assistance, 
and less than half ever receiving Food Stamps. Table 6.4 presents the four-year and annual impacts 
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TABLE 6.3 


IMPACTS ON AFDC, FOOD STAMPS, AND GENERAL ASSISTANCE THROUGH MONTH 48, 
FOR WOMEN LIVING WITH THEIR OWN CHILD(REN) AT THE TIME OF RANDOM ASSIGNMENT 


Outcome and Follow-Up Period Experimentals | Controls Difference p* 
Ever received AFDC (%) 

Years 1-4 84.8 81.6 3.2 0.318 
Ever received AFDC (%) 

Year 1 65.6 61.2 4.4 0.262 

Year 2 75.5 74.3 1.2 0.737 

Year 3 57.6 57.5 0.5 0.919 

Year 4 59.3 60.5 1.2 0.801 
Total AFDC income ($) 

Years 1-4 9,371 9,334 37 0.952 
AFDC income ($) 

Year 1 2,167 2,072 95 0.558 

Year 2 2,402 2,279 123 0.471 

Year 3 2,310 2,343 33 0.874 

Year 4 2,493 2,641 -148 0.515 
Ever received Food Stamps (%) 

Years 1-4 84.5 88.3 3.9 0.195 
Ever received Food Stamps (%) 

Year 1 60.9 63.4 -2.5 0.504 

Year 2 71.3 77.5 -6.1* 0.093 

Year 3 71.1 64.1 7.0* 0.078 

Year 4 74.6 71.6 3.0 0.442 
Total Food Stamp income (S$) 

Years 1-4 4,872 4,760 112 0.710 
Food Stamp income ($) 

Year 1 849 947 -98 0.201 

Year 2 982 1,090 -108 0.195 

Year 3 1,401 1,251 150 0.163 

Year 4 1,639 1,472 167 0.154 
Sample size 249 242 

(continued) 
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TABLE 6.3 (continued) 


. 


Outcome and Follow-Up Period Experimentals Controls Difference p* 
Ever received General Assistance (%) 

Years 1-4 13.6 12.9 0.7 0.815 
Ever received General Assistance (%) 

Year 1 6.8 6.6 0.2 0.938 

Year 2 8.3 7.5 0.8 0.713 

Year 3 6.9 4.0 29 0.153 

Year 4 8.1 5.3 27 0.227 
Total General Assistance income (S$) 

Years 1-4 744 385 359°* 0.025 
General Assistance income (S) 

Vear 1 205 176 29 0.656 

Year 2 79 36 43 0.192 

Year 3 210 87 123° 0.051 

Vear 4 249 85 164** 0.018 
Sample size 249 242 


SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 


NOTES: Calculations for this table used data for all sampie members for whom there were 
48 months of follow-up survey data on the specific outcomes, including those with values of zero for 
outcomes and those who were assigned to JOBSTART but did not participate. 

Random did not always take place on the first of the month. For some sample 


average experimental and control group outcomes. Statistical significance levels are indicated as *** 
= 1 percem; ** = 5 percent; * = 10 percent. 
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TABLE 6.4 


THROUGH MONTH 48, FOR MEN 


Outcome and Follow-Up Period Experimentals Controls 
Ever received AFDC (%) 

Years 1-4 6.2 6.2 
Ever received AFDC (%) 

Year 1 1.2 1.6 

Year 2 1.8 2.6 

Year 3 3.7 3.4 

Year 4 4.9 3.1 
Total AFDC income (S$) 

Years 1-4 386 334 
AFDC income (S$) 

Year 1 35 34 

Year 2 27 7 

Year 3 153 106 

Year 4 170 118 
Ever received Food Stamps (%) 

Years 1-4 43.3 41.7 
Ever received Food Stamps (%) 

Year 1 25.1 26.0 

Year 2 32.4 29.9 

Year 3 19.5 18.6 

Year 4 24.4 23.2 
Total Food Stamp income ($) 

Years 1-4 1,423 1,333 
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IMPACTS ON AFDC, FOOD STAMPS, AND GENERAL ASSISTANCE 


Difference 


0.0 


0.132 
0.357 


TABLE 6.4 (continued) 


Outcome and Follow-Up Period Experimentals Controls Difference p* 
Ever receivec General Assistance (%) 

Years 1-4 10.5 7.9 2.6 0.151 
Ever received General Assistance (%) 

Year 1 6.1 29 3.2°* 0.014 

Year 2 5.3 4.9 0.4 0.773 

Year 3 2.9 2.9 -0.1 0.961 

Year 4 4.0 3.6 0.4 0.752 
Total General Assistance income ($) 

Years 1-4 234 268 34 0.622 
General Assistance income ($) 

Year 1 100 56 4a* 0.092 

Year 2 46 §9 13 0.587 

Year 3 37 70 33 0.195 

Year 4 51 83 32 0.272 
Sample size 434 432 


SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 
NOTES: Calculations for this table used data for all sample members for whom there were 


Random assignment did not always take place on the first of the month. For some sample 
members, the month of random assignment may be less than a month, beginning with the date of 
random assignmont and ending on the last day of the month. 

Average experimental and contro! group outcomes reported here are adjusted means from 
linear analysis of covariance procedures controlling for up to 29 kinds of difference in characteristics 
before random assignment (see Ostle, 1975, p. 461; Cave, 1987; and Appendix B). There may be 


average experimental and control group outcomes. Statistical significance levels are indicated as *** 
= 1 percent; ** = 5 percent; * = 10 percent. 


on welfare receipt rates and payments for young men. JOBSTART had almost no significant® or 
substantial impacts on these three types of welfare receipt and income for this subgroup. 


B. Impacts on Receipt of Other Public Benefits 


Table 6.5 shows that for the full sample JOBSTART had no significant impacts during the four- 
year follow-up period on the receipt of public housing, Heat Assistance, Unemployment Insurance 
benefits, Medicaid care, or Supplemental Security Income (SSI). There was a statistically significant 
but small impact on receipt of Workers’ Compensation: 4.7 percent of controls received this benefit 
during follow-up, compared to 3.1 percent of experimentals. There were no notable exceptions to 
this general pattern among the three key subgroups (not shown in the table). 


Young women who delay starting a family generally have a greater chance of acquiring a higher 
level of education and consequently enhancing their opportunities in the labor market than their 
peers who have children. Further, young women who are unmarried and become mothers have a 
higher probability of being dependent on welfare. Over half of all AFDC expenditures go to 
maintain households in which the mother was a teenager when her first child was born. In 1990, 
taxpayers spent an estimated $25 billion, an increase of $3.5 billion from 1989, on mothers who 
began families in their teens.’ These expenditures were in the form of AFDC, Food Stamps, and 
Medicaid payments. Society stands to benefit if pregnancy and childbearing among young women 
ste postponed, since this postponement may lead to an overall increase in their educational 
attainment and skills needed to help them escape from poverty. 

Table 6.6 summarizes the four-year impacts of JOBSTART on pregnancy and births for women 
who were custodial mcthers at random assignment and for women who were not (“other women" 
in the table). Among all custodial mothers, the rates of pregnancy and childbirth (shown in the top 
panel of the table) were higher for experimentals than controls, and the differences were large and 
statistically significant. It is important to note that the birth rate for controls was itself high: 57.9 
percent of custodial mothers in the control group had a subsequent birth during follow-up, compared 
to 67.8 percent of experimentals. Looking behind the higher birth rate for experimentals among 


©The only exceptions were a significant 3.2 percentage point experimental-control difference in the rate 
of General Assistance receipt in the first year after random assignment, and a significant $44 difference in 
General Assistance income in that year. 

7Center for Population Options, 1992. 
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TABLE 6.5 


IMPACTS ON OTHER PUBLIC BENEFITS THROUGH MONTH 48 


Experimentals Controls 


Outcome and Follow-Up Period (%) (%) Difference p* 
Ever received public housing 

Years 1-4 30.8 30.6 0.1 0.946 
Ever received Workers’ Compensation 

Years 1-4 3.1 47 -1.6* _ 0.071 
Ever received Heat Assistance 

Years 1-4 21.7 22.3 -0.6 0.718 


Ever received Unemployment insurance 
benefits 


Years 1-4 8.2 7.6 0.6 0.631 
Ever received a Medicaid card 

Years 1-4 57.2 §5.5 1.7 0.372 
Ever received Su, plemental Security 
income (SSI) 

Years 1-4 1.8 26 0.8 0.260 


SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 
NOTES: Calculations for this table used data for all sample members for whom there were 


Random assignment did not always take place on the first of the month. For some sample 
members, the month of random assignment may be less than a month, beginning with the date of 
month. 


average experimental and control group outcomes. Statistical significance levels are indicated as *** 
= 1 percent; ** = 5 percent; * = 10 percent. 
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TABLE 6.6 


IMPACTS ON PREGNANCY AND CHILDBIRTH THROUGH MONTH 48, 
BY PARENTAL STATUS AT THE TIME OF RANDOM ASSIGNMENT 


Subgroup, Outcome, 


and Follow-Up Period Experimentals Controls Difference p* 

All custodial mothers at 

random assignment 
Ever pregnant, years 1-4 (%) 76.1 67.5 8.6** 0.032 
Ever gave birth, years 1-4 (%) 67.8 57.9 9.9°* 0.024 
Sample size 255 249 

Custodial mothers never married 

at random 
Ever pregnant, years 1-4 (%) 75.2 70.5 4.7 0.296 
Ever gave birth, years 1-4 (%) 66.5 60.0 65° 0.178 
Sample size 208 198 

Custodial mothers ever married 

at random assignment 
Ever pregnant, years 1-4 (%) 79.6 56.1 23.5** 0.037 
Ever gave birth, years 1-4 (%) 73.0 50.3 22.7* 0.066 
Sample size 47 51 

Other women 
Ever pregnant, years 1-4 (%) 64.4 65.6 “1.2 0.781 
Ever gave birth, years 1-4 (%) 52.7 56.5 3.9 0.380 
Sample size 278 247 


SOURCE: MDRC calculations from JOBSTART enroliment form and survey data. 
NOTES:  Caiculations for this table used data for all sample members for whom there were 48 


Random assignment did not always take place on the first of the month. For some sample 
members, the month of random assignment may be less than a month, beginning with the date of 
random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted means from 
linear analysis of covariance procedures controlling for up to 29 kinds of difference in characteristics 


sligm discrepancies in reported sums and differences adjusted means because of 
"The column labeled °p’ is the statistical level of the difference 
experimental and contro! group outcomes: that is, p is the probability that outcomes are 


between 
average experimental and control group outcomes. Statistical significance levels are indicated as *** 
= 1 percent; ** = & percent; * = 10 percent. 
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custodial mothers, the two middle panels of the table reveal that the birth rate impact was largest 


for custodial mothers who were married at random assignment, though it was also positive (though 
smaller and not significant) for those who were unmarried. The experimental-control differences 
of 23.5 and 22.7 percentage points for pregnancy and childbirth rates, respectively, for married 
custodial mothers were statistically significant. Despite these large impacts, because of the relatively 
small number of women who were married custodial mothers at random assignment, almost two- 
thirds of births to all custodial mothers during the follow-up period occurred among unmarried 
custodial mothers. 

Among women who were not custodial mothers at random assignment (shown in the bottom 
panel of the table), 56.5 percent of the control group had given birth by the end of the follow-up 
period, compared to 52.7 percent of the experimental group, a difference that was not statistically 
significant. There was also a small and statistically insignificant reduction in the pregnancy rate for 
this subgroup. 

These varying impacts on birth rate for the two subgroups of women are rzlated to the findings 
already presented about impacts on AFDC receipt and income. One likely reason for the iower 
AFDC income of experimentals among “other women" is their slightly lower rate of childbirth as 
compared to controls, coupled with positive earnings impacts in later years. (AFDC grants decline 
as income increases.) Among custodial mothers, the higher birth rate for experimentals balanced 
out their positive earnings impacts, leading to no impact on AFDC income. 

On the related issue of marital status (not shown in the table), JOBSTART had no impact on 
the proportion of young people who had ever married by the end of the four-year follow-up period. 
Approximately 75 percent of both the experimental and control groups remained unmarried, with 
slightly more than 15 percent of both groups married and living with a spouse by the end of follow- 
up. There were no impacts on marital status for either subgroup of women; for males, experi- 
mentals were slightly more likely to have gotten married, though the difference was insignificant. 


III. Impacts on Criminal Activity and Drug Use 


Currently there is great public concern about criminal activity and drug use, especially among 
low-income males. The JOBSTART Demonstration provided an opportunity to learn more about 
the levels of criminal activity and drug use among low-income young people, as well as the impact 
of an education and training program on such activities. Questions about criminal activity are 
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relevant when analyzing the JOBSTART sample of disadvantaged youths, especially the young men, 
since almost half of the males were arrested during the four-year follow-up period.® 


A. Criminal Activity 


In evaluating a job training program such as JOBSTART, one might hypothesize that when 
opportunities for participants to be employed become greater, involvement in illegal activities 
becomes less attractive. The expectation is that fewer crimes and fewer arrests will occur, reducing 
the burden on the judicial system and taxpayers. , 

Table 6.7 presents impacts on arrests in the first year of follow-up and over the entire four 
years. JOBSTART produced a small but significant reduction in arrests for the full sample during 
the first year of follow-up (2.6 percentage points), when most experimentals were active in the 
program, but there was no statistically significant impact over the entire four-year period. The 
pattern for the key subgroups shown in the table was in most cases similar: a small (and in one case 
statistically significant?) in-program decline in arrests, followed by a post-program period when the 
impact disappeared. The exception to this pattern is an important one: For males who had been 
arrested between age 16 and random assignment, the in-program impact was virtually zero, and in 
the remainder of the follow-up period controls appeared to be somewhat more likely to be arrested 
than experimentals.!° 

These impacts were more modest than those found in the previous study of the residential Job 
Corps program. Males in the Job Corps study experienced a 9.4 percentage point lower arrest rate 
while in the program than their comparison group, while females had a 2.5 percentage point lower 
rate during participation.!! However, much of the Job Corps’ impact on criminal behavior during 
the in-program period probably resulted from the “isolation” effect. In the residential Job Corps 
program, young people move from their community to a special center that provides the education 
and training services. These centers are often in isolated areas or communities without a large gang 
population or heavy drug trade activity. In JOBSTART, the young people continued to live in their 
neighborhood and spent time outside the program with their existing circle of friends. 


Sin all the follow-up surveys, information was collected on arrests rather than convictions. This was done 
because of the frequency of plea bargains, the special treatment of young offenders by the courts, and the time 
lag between arrests and trials in many jurisdictions. The surveys, therefore, did not indicate whether arrests 
led to conviction or incarceration. 

*The in-program impact for males without a prior arrest was 6.4 percentage points and statistically 
significant. 

Mo. peared to be" because the sample size was too small to allow the difference to be statistically 
significant, although the absolute difference was 5.8 percentage points, or slightly over 8 percent. 

Mallar et al., 1978. 
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TABLE 6.7 


IMPACTS ON ARRESTS THROUGH MONTH 48 
FOR THE FULL SAMPLE AND KEY SUBGROUPS 


Subgroup, Outcome, 


and Follow-Up Period Experimentals § Controls Difference 

Full sample 
Ever arrested, year 1 (%) 10.1 12.6 -2.6* 0.092 
Sample size 851 779 
Ever arrested, years 1-4 (%) 29.0 29.3 -0.2 0.902 
Sample size 982 942 

Men not arrested between age 16 

and random assignment 
Ever arrested, year 1 (%) 11.2 17.6 -6.4** 0.037 
Sample size 277 278 | 
Ever arrested, years 1-4 (%) 37.6 38.4 0.8 0.830 
Sample size 445 448 

Men arrested between age 16 

and random assignment 
Ever arrested, year 1 (%) 35.1 35.1 0.1 0.992 
Sample size 106 82 
Ever arrested, years 1-4 (%) 68.9 74.8 5.8 0.362 
Sample size 127 109 

Women living with own child(ren) 
Ever arrested, year 1 (%) 3.4 6.3 -2.9 0.167 
Sample size 221 216 
Ever arrested, years 1-4 (%) 14.3 16.0 “1.7 0.594 
Sample size 256 247 


Women not living with own child(ren), 
including those who did not have any 


Ever arresied, year 1 (%) 3.5 4.2 0.7 0.696 
Sample size 247 203 
Ever arrested, years 1-4 (%) 12.7 12.3 0.4 0.880 
Sample size 281 247 
(continued) 
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TABLE 6.7 (continued) 


SOURCE: MODRC calculations from JOBSTART enrollment form and survey data. 


NOTES: Calculations for this table used data for all sample members for whom there were 
48 months of follow-up survey data on the specific outcome, including those with values of zero for 
outcomes and those who were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For some sample 
members, the month of random assignment may be less than a month, beginning with the date of 
random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted means from 
linear analysis of covariance procedures controlling for up to 31 kinds of difference in characteristics 
before random assignment (see Ostle, 1975, p. 461; Cave, 1987; and Appendix B). There may be 
slight discrepancies in reported sums and differences of these adjusted means because of rounding. 

*The column labeled 'p’ is the statistical significance level of the difference between 
experimental and control group outcomes: that is, p is the probability that average outcomes are 
different only because of random error. A two-tailed t-test was applied to each difference between 
average experimental and control group outcomes. Statistical significance levels are indicated as *** 
= 1 percent; ** = 5 percent; * = 10 percent. 


B. Drug Use 


Drug use among young people continues to have serious repercussions, although surveys 
indicate that the percentage of the U.S. population that has tried drugs has remained relatively 
constant during the past 15 years.'? Many of the arrests discussed in the previous section were 
drug-related offenses, and this type of criminal activity can have long-lasting effects. Drug use among 
adolescents has been correlated with low educational achievement, a higher rate of infection with 
the AIDS virus, and an increase in violent crime.!* 

Table 6.8 provides data on drug use by experimentals and controls in the JOBSTART 
Demonstration during the final year of follow-up.'4 For that year, slightly fewer experimentals 
(16.2 percent) than controls (17.5 percent) reported ever using any drugs, but this difference was 
not statistically significant. More controls than experimentals reported the use of drugs other than 
marijuana, and the 1.7 percentage point difference was statistically significant. 

Reported drug use was consistently higher among males than females: For example, more than 
20 percent of males rcported use of any drug in the year prior to the final survey, compared to 12 
percent of females. JOBSTART led to a 2.5 percentage point decline in the use of any drug in the 
prior year for males (not significant), compared to no impact for females. For drugs other than 
marijuana, males experienced a 1.7 percentage point decline and females a 1.3 percentage point 
decline. For marijuana, impacts were -2.7 percentage points for males and -0.9 percentage points 
for females. 

Impacts on drug use were strong and, in one case, significant for males with an arrest between 
age 16 and entry into JOBSTART. For this group, drug use was much higher than for the rest of 
the sample — over one-fourth reported using drugs in the prior year. Twenty-five percent of 
experimentals reported using any drug in the prior year, as compared to 31 percent of controls, for 
a difference of 5.5 percentage points (not significant). Focusing on drugs other than marijuana, 3.7 
percent of experimentals and 10.5 percent of controls reported use in the prior year, resulting in a 
large (6.9 percentage points or 186 percent) and significant impact. Marijuana use also appeared 
to be lower among experimenials (by 4.9 percentage points), but again the impact was not 
statistically significant. > 


12U.S. General Accounting Office, 1991. 
134.§. General Accounting Office, 1991. 
14in the final survey, youths were asked about drug use during the prior year. 
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TABLE 6.8 


YEAR-FOUR IMPACTS ON DRUG USE FOR 
THE FULL SAMPLE AND KEY SUBGROUPS 


Subgroup, Outcome, 


and Follow-Up Period Experimentals Controls Difference p* 
Full sample 
Ever used any drug in year 4 (%) 16.2 17.5 -1.2 0.469 
Ever used any drug in year 4, 
excluding marijuana (%) 41 5.8 -1.7* 0.093 
Ever used marijuana in year 4 (%) 14.6 16.3 -1.8 0.279 
Sample size 980 938 
All men 
Ever used any drug in year 4 (%) 21.2 23.7 2.5 0.380 
Ever used any drug in year 4, 
excluding marijuana (%) 4.6 6.3 1.7 0.264 
Ever used marijuana in year 4 (%) 20.0 22.7 2.7 0.326 
Sample size 440 445 
Men arrested between age 16 
and random assignment 
Ever used any drug in year 4 (%) 25.4 31.0 5.5 0.386 
Ever used any drug in year 4, 
excluding marijuana (%) 3.7 10.5 -6.9** 0.058 
Ever used marijuana in year 4 (%) 25.3 30.2 4.9 0.437 
Sample size 123 109 
All women 
Ever used ary drug in year 4 (%) 12.0 12.0 0.0 0.994 
Ever used any drug in year 4, 
excluding marijuana (%) 3.9 5.3 1.3 0.307 
Ever used marijuana in year 4 (%) 9.9 10.8 -0.9 0.637 
Sample size 540 493 


SOURCE: MDRC calculations from JOBSTART enroliment form and survey data. 


NOTES: Calculations for this table used data for all sample members for whom there were 
48 months of follow-up survey data on the specific outcomes, including those with values of zero for 
outcomes and those who were assigned to JOBSTART but did not participate. 

Random assignment did not always take place on the first of the month. For some sample 
members, the month of ranciom assignment may be less than a month, beginning with the date of 
random assignment and ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted means from 
linear analysis of covariance procedures controlling for up to 31 kinds of difference in characteristics 
before random assignment (see Ostle, 1975, p. 461; Cave, 1987; and Appendix B). There may be 
sligm discrepancies in reported sums and differences of these adjusted means because of rounding. 

*The column labeled °p’ is the statistical significance level of the difference between 
experimental and control group outcomes: that is, p is the probability that average outcomes are 
different only because of random error. A two-tailed t-test was applied to each difference between 
average experimental and control group outcomes. Statistical significance levels are indicated as *** 
= 1 percent; ** = 5 percent; * = 10 percent. 
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IV. Impacts on Positive Activity 


A further step in understanding JOBSTART’s employment and earnings impacts is to examine 
whether access to the program led more experimentals than controls to spend their time in “positive 
activity,” defined as either work or further education and training. This outcome measure combines 
the findings from participation in education and training and employment into one composite 
measure. 

As already discussed, experimentals initially were working less than controls because they were 
active in JOBSTART, and even in the later years of follow-up, continued investment in education 
and training by experimentals could have delayed employment "payoffs." Table 6.9 presents impacts 
on positive activity for the full sample, men, custodial mothers, and all other women. For the full 
sample and the three subgroups, over the entire 48-month follow-up period, nearly all individuals 
in the experimental and control groups had done something “positive” at some point, although 
experimentals had slightly higher rates of such activity. During the first 24 months of follow-up, a 
higher percentage of experimentals than controls spent time in positive activity, with a statistically 
significant difference for the full sample and each subgroup. But by the end of the first 24 months 
of follow-up, monthly percentages of experimentals and controls engaged in positive activity were 
approximately equal (not shown in the table). Over the final two years of follow-up, almost equal 
numbers of experimentals and controls were in either employment or education and training 
activities. 


Vv. A Summary of JOBSTART’s Subgroup Impacts 


In this section, the impact findings presented in Chapters 5 (on labor market outcomes) and 
6 (on non-labor market outcomes) are summarized for custodial mothers, all other women, all males, 
and males arrested between age 16 and program entry. Again, it is important to remember that 
dividing the sample into these subgroups reduces the likelihood of finding statistically significant 
impacts. 

For the two female subgroups, JOBSTART appears to have made the most differences in the 
lives of women who were not custodial mothers when they entered the program. Positive earnings 
impacts, while not statistically significant, did appear to be present in the third and fourth years of 
follow-up and for the entire four-year period. Pregnancy and childbirth rates were slightly lower for 
experimentals than controls for this group, and AFDC receipt was clearly lower. Since all of these 
young women, as low-skilled school dropouts, were at risk of serious employment problems and 
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TABLE 6.9 


IMPACTS ON POSITIVE ACTIVITY THROUGH MONTH 48, 
BY GENDER AND PARENTAL STATUS 


Subgroup, Outcome, 


and Follow-Up Period Experimentals Controls Difference p* 

Full sample 
Positive activity, years 1-2 (%) 97.6 85.5 12,4°** 0.000 
Positive activity, years 3-4 (%) 79.9 80.9 -1.0 0.558 
Positive activity, years 1-4 (%) 98.6 93.9 4.7*** 0.000 
Sample size 988 953 

Men 
Positive activity, years 1-2 (%) 98.6 92.1 6.6*** 0.000 
Positive activity, years 3-4 (%) 89.0 89.2 -0.2 0.920 
Positive activity, years 1-4 (%) 99.5 97.2 2.3*** 0.007 
Sample size 448 452 

Women living with own child(ren) 
Positive activity, years 1-2 (%) 95.5 73.9 21.70%" 0.000 
Positive activity, years 3-4 (%) 69.9 71.8 -1.9 0.638 
Positive activity, years 1-4 (%) 97.5 89.4 5.15** 0.000 
Sample size 257 251 


Women not living with own child(ren), 
including those who did not have any 


Positive activity, years 1-2 (%) 97.5 85.6 +1.o°"* 0.000 
Positive activity, years 3-4 (%) 74.9 74.8 0.1 0.980 
Positive activity, years 1-4 (%) 97.9 92.8 5.28 0.005 
Sample size 283 250 


SOURCE: MDRC calculations from JOBSTART enroliment form, MIS, and survey data. 


NOTES: Calculations for this table used data for all sample members for whom there were 
48 months of follow-up survey data on the specific outcomes, including those with values of zero for 
outcomes and those who were assigned to JOBSTART but did not participate. 


before random assignment ( , 
irk Gactepatcies 1 renoned sume and Cilererces of hese adhusied teins bachuse £1 (oUnding: 
*The column labeled ‘p’ is the statistical significance level of the difference between 
experimental and control group outcomes: that is, p is the probability that average outcomes are 
differem only because of random error. A two-tailed t-test was applied to each difference between 
average experimental and control group outcomes. Statistical significance levels are indicated as *** 
= 1 percent; ** = § percent; * = 10 percent. 
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subsequent welfare receipt, these findings are encouraging signs that JOBSTART could serve as a 


welfare prevention program. 

The impact story for custodial mothers and males is less encouraging. For custodial m« thers, 
there were positive — though not significant — earnings impacts in the third and fourth years and 
for the entire four-year period, but higher pregnancy and childbirth rates for experimentals and no 
decline in welfare receipt. For males, experimentals remained behind controls in four-year earnings 
because of large initial losses during the in-program period. Furthermore, there were few 
encouraging impact findings on the other outcomes analyzed in the chapters. 

However, for males with a prior arrest the findings are encouraging. Earnings impacts were 
large and significant, and began to appear early in the follow-up period. Further, there were large 
and significant impacts on drug use and some signs that JOBSTART may have reduced post- 
program arrests. Although caution is in order when reviewing these findings owing to the small 
sample size for this subgroup, JOBSTART appears to have made a clear positive difference for 
these young men, whose serious barriers to employment and troubled lives are a matter of great 
public concern. 
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CHAPTER 7 


0 N OB 


This chapter presents a comparison of the costs and benefits of the JOBSTART program. It 
draws on program cost information presented in Appendix C and on the impact data in Chapters 
5 and 6 of this report. The benefits and costs are assessed from the perspectives of program 
participants, taxpayers, and society as a whole.! This analysis attempts to summarize the main 
program impacts discussed in earlier chapters and to provide a framework for answering several 
fundamental questions regarding JOBSTART's achievements. Within the benefit-cost framework, 
the benefits produced by JOBSTART's addition of services above those received by controls are 
compared to the cost of these added services. 

Among other topics, this chapter examines whether the added services provided by JOBSTART 
had a net payoff for participants, and if so, when during the follow-up period such a payoff began. 
In calculating net program effects, participants’ earnings, welfare benefits, and program-related 
transfer payments are combined into a summary measure, describing the program’s overall measured 
economic consequences for participants. This analysis is done for JOBSTART’s full 48-month 
impact sample, as well as for the three key subgroups: men, women who were custodial mothers 
at entry into the program, and all other women in the sample. 

An important aspect of any benefit-cost analysis is the influence of time on the final 
conclusions about cost-effectiveness. The period considered in this analysis begins at the point when 
participants and funders invested their resources, and extends over the four-year follow-up period. 
For the initial investment to pay off, future benefits must exceed it by a certain margin, representing 
the time value of money and the effect of inflation. By discounting benefits to the initial in-program 
year, the benefit-cost analysis addresses the question of the program's real value over time, in this 
case the four-year follow-up period. 

It must be noted that many questions remain unanswered in this benefit-cost analysis. In its 


1 The taxpayer perspective is not entirely appropriate in the context of the JOBSTART Demonstration, 
which was funded in part with private contributions. However, if JOBSTART was expanded into a more large- 
scale program, taxpayers would probably be the primary funders and benefits would accrue to them. The 
funders’ perspective is therefore referred to in this report as the taxpayers’. Given certain assumptions 
regarding the distribution of costs and benefits — especially that benefits and costs to participants and 
taxpayers should be weighted equally — the taxpayer and participant perspectives can be combined into a 
“societal” perspective. 
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attempt to combine different program effects into a summary measure, the analysis excludes many 


program effects that cannot be expressed in dollar terms. For instance, in Chapter 4, JOBSTART 
was shown to have substantial impacts on the receipt of education services and subsequent 
educational attainment. Aside from their indirect effects on earnings and welfare receipt, these 
education impacts are not represented in this analysis, even though additional, non-monetary 
benefits for particinants and taxpayers may be associated with these education gains. 

The first of the six sections in this chapter describes the analytical framework. The second 
begins by presenting the program's gross costs? and continues with a discussion of the receipt of 
non-JOBSTART services by the control and experimental groups, and an estimation of the net cost 
of JOBSTART. Section III presents costs and benefits from the perspective of program 
participants. These costs and benefits are based on the impacts discussed in Chapters 5 and 6 and 
the cost analysis in Section II. Section IV includes a benefit-cost comparison from the taxpayers’ 
perspective and concludes with an analysis that combines aspects of both participant and taxpayer 
perspectives to determine the payoff of JOBSTART to society as a whole. Some of the non- 
monetary effects of JOBSTART are considered in Section V in relation to the benefit-cost analyses 
presented earlier, and the chapter concludes in Section VI with a summary. 


I. Analytical Approach 


This benefit-cost analysis uses an approach similar to that followed in MDRC’s previous 
evaluations of job training and welfare employment programs.> The analysis combines the dollar 
values of the program's measured effects and its use of resources into a net present value. Non- 
monetary program effects can be considered in comparisons of the program's benefits and costs. 
Once monetary benefits and costs have been calculated and compared, the reader can assess 
whether non-monetary factors are likely to be large enough to change the initial conclusion. The 
effects and resources included in this analysis are listed in Table 7.1, which also indicates their 
expected general value (positive, negative, or zero) from the three perspectives considered. 

The resources include all JOBSTART operating costs, regardless of funding source. They also 
include compensation to participants for program-related expenses as well as the cost of support 
services. Both experimentals and controls received non-JOBSTART education and training services 
during the four-year follow-up period. The effect of the use of these non-JOBSTART resources 


2s mentioned above, this analysis uses Appendix C as its primary data source. 
3See, for instance, Hamilton and Friedlander, 1989, p. 91. 
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TABLE 7.1 


THEORETICAL VALUE OF COMPONENTS OF THE JOBSTART BENEFIT-COST ANALYSIS, 
BY ACCOUNTING PERSPECTIVE 


Accounting Perspective 


Program 
Component Participants Taxpayers Society 
Increased earnings and fringe benefits + 0 + 
Increased tax payments 
Payroll taxes 
Income and sales taxes 
Reduced use of transfer programs 
AFDC payments - + 0 
Food Stamp payments - + 0 
General Assistance payments - + 0 
Payments from other public programs - + 0 
AFDC administrative costs 0 + + 
Food Stamp administrative costs 0 + + 
Reduced use of community education 
and training programs 0 + + 
Reduced criminal activity and income - + + 
JOBSTART operating costs 0 - - 
Compensation for program-related expenses 0 - - 
Additional support services + - 0 
Change in family’s financial needs owing to 
program effects on childbearing ? ? ? 
Value of education not reflected in earnings + + + 
Preference for work over welfare + + + 
Foregone leisure time and activities - 0 - 


NOTE: The components are shown as a theoretical benefit (+), cost (—), or neither a benefit 
nor a cost (0), according to a priori expectations regarding their value. 
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on the net program costs is discussed in Section IIB below. The monetary program impacts 


considered include those on earnings and welfare receipt, discussed in Chapters 5 and 6, as well as 
effects on tax payments, Food Stamps, and administrative costs associated with transfer programs. 
Table 7.1 also includes non-monetary benefits such as the value of education not reflected in 
earnings and a preference for work over welfare, and non-monetary costs such as foregone leisure 
time. 

Whether a given program effect or the use of specific resources is a benefit or a cost depends 
on what is actually measured and the analytical perspective that is taken. Table 7.1 shows 
components of the benefit-cost analysis and the expected value of each component from the 
participant, taxpayer, and societal perspectives. Each plus and minus in Table 7.1 reflects the 
expected benefit or cost status of a component assuming that the program is successful, but the 
actual result may be different. Once measured, particular effects or expenses are valued as a 
benefit, a cost, or irrelevant, depending on which of the three perspectives is considered. 

The program participants’ perspective identifies the benefits and costs for members of the 
experimental group, indicating how they fared as a result of JOBSTART. This perspective is 
particularly important in a comparison of program benefits to opportunity costs for participants. The 
taxpayers’ perspective identifies benefits and costs from the standpoint of everyone in society other 
than individuals who received program services. For example, if JOBSTART reduced AFDC receipt 
among its participants, that effect would translate into a loss for participants but a gain for taxpayers. 
The societal perspective assesses whether the program's benefits exceed its costs when the 
participant and taxpayer perspectives are combined. 


Tl. Program Costs 


An extensive analysis of JOBSTART program costs was performed to identify the market value 
of all resources used in providing JOBSTART services. It therefore includes as program costs not 
only the expenses incurred by the agencies sponsoring the program, but also those incurred by 
outside organizations responsible for providing certain services, such as occupational skills training 
in the three sequential/brokered sites. 

During the JOBSTART Demonstration, data were collected from a variety of sources 
containing information on fixed and variable operating costs, support services, and participation 
rates. In most cases, data from these sources covered a one-year “steady-state” period between 1985 
and 1988, the years during which JOBSTART was funded. The year-long period selected in most 
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sites began at least several months after the initiation of the project (in order to avoid including the 
Start-up costs associated with beginning a new program) and ended at least several months prior to 
the termination of the demonstration (in order to exclude the phaseout period). 


A. The Gross Costs of JOBSTART 


The average total cost of JOBSTART per experimental was determined by summing the 
average cost of several relatively distinct program components and services (see Table 7.2, which 
contains weighted averages for all sites).4 The table shows that basic education was the largest cost 
component of the program, with over 22 percent of total expenses, followed by occupational skills 
training, with 20 percent of total expenses. Another 19 percent of program expenses were dedicated 
to program coordination anc counseling. The fact that the program focused primarily on human 
capital development is apparent in the relatively low level of spending on labor force attachment 
services, such as job development and work-readiness skills training, which together made up less 
than 13 percent of the total expenses. 

The average cost per experimental for the delivery of the JOBSTART services (intake, 
education, training, job placement assistance, counseling, and life skills workshops) varied widely 
across the sites (see Table 7.3, under “Service Delivery"). Although it fell between $4,000 and 
$5,500 in most sites, it ranged from slightly over $2,000 in CET/San Jose, EGOS in Denver, and 
SER/Corpus Christi to a high of about $7,500 in BSA in New York City.5 Calculating the cost 
per participant instead of per experimental can change the picture somewhat. In particular, the cost 
per experimental at CET/San Jose was substantially lower than the cost per participant at the site 
because only 64 percent of experimentals there were ever active in JOBSTART, as compared to the 
all-site average of 89 percent. Consequently, on a per-participant basis, EGOS in Denver had the 
lowest service delivery cost among the sites. 

As discussed in Chapter 3, JOBSTART also offered support services and small stipends, which 
varied by site and were for transportation, child care, and other services and expenses related to 
participation in JOBSTART. The total cost of these benefits varied from a low of $61 at CET/San 
Jose to a high of $1,389 at the Los Angeles Job Corps site. In the majority of sites, the payments 
and support services averaged between $200 and $600 per experimental. In Table 7.3, this category 


“The figures in Table 7.2 were calculated by multiplying the expenses per experimental reported by the 
sites for each program component by the number of experimentals in the site, and dividing the total by 988 
(the number of experimentals in the 48-month sample). 

SFor a discussion of the sources of these figures, see Appendix C. 
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TABLE 7.2 
COSTS OF JOBSTART, BY PROGRAM COMPONENT 


As a % 
Component In$ of the Total Cost 
Service delivery 
Honckpollnchibeis intake, and orientation 496 10.9 
1,014 22.3 
coahenrere skills training 915 20.1 
Work-readiness or life skills training 328 7.2 
Job development and placement assistance 235 5.2 
Counseling and program coordination 875 19.2 
Support services 568 12.5 
Subtotal for core components 4,431 97.4 
Medical/dental services 117 2.6 
Total 4,548 100.0 


SOURCE: Appendix C. 


NOTES: The cost estimates are based on a sample of 949 experimentals for whom there 
were 24 rvonihs of follow-up survey data in 1991. These estimates are weighted averages of the 
site-specific figures, caiculated by dividing the sum-product of these figures by 988, the number of 
experimentals for whom there were 48 months of follow-up survey data. 

All costs are in 1986 dollars. 
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TABLE 7.3 


COSTS OF JOBSTART, BY SITE AND PROGRAM COMPONENT 


Concurrent 
Atlanta Job Corps 
CET/San Jose 
Chicago Commons 
Connelley (Pittsburgh) 
East LA Skills Center 
EGOS (Denver) 
Phoenix Job Corps 
SER/Corpus Christi 


Sequential/in-house 
El Centro (Datias) 
LA Job Corps 


Sequential/brokered 
Allentown (Buffalo) 
BSA (NYC) 

CREC (Hartford) 


All sites 


SOURCE: Appendix C. 


NOTES: The cost estimates are based on a sample of 949 experimentals for whom there were 24 munths of 
follow-up survey data in 1991. The ‘all sites* estimates are weighted averages of the site-specific figures, calculated by 
dividing the sum-product of these figures by 968, the number of experimentals for whom there were 48 months of follow-up 

a2 survey data. 

All costs are in 1986 dollars. 
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of expenses is presented separately from service delivery costs anc medical/dental costs to allow 
them to be treated differently depending on the analytical perspective.© In addition to the usual 
support services, some sites provided extensive medical benefits,’ which must be considered as 
transfer payments from taxpayers to participants. A more extensive discussion of program costs, and 
a detailed breakdown by site of the costs of services provided, is included in Appendix C. 

As shown in Table 7.3, the average total cost per JOBSTART experimental was $4,548,° of 
which $3,863 (85 percent) was spent to operate the program, $568 (12 percent) on support services, 
and $117 (3 percent) on medical and dental services.” In the United States, the average annual 
cost per pupil for public education was $3,839 for the 1985-86 school year,!° the first year of the 
JOBSTART program operation. JOBSTART was a remedial program for high school dropouts and 
can be considered an attempt to offset the negative effects of missing two school years, or the 
equivalent of a $7,600 public investment.!! Other programs serving a comparable target group 
are JTPA and the Job Corps. The former, which is less intensive than JOBSTART, spent less than 
$2,000 per new enrollee in program year 1986.'? The latter, which offers more intensive services 
than either JTPA or JOBSTART, including providing residential services in addition to education 
and training, cost $10,545 per year per participant in the mid-1980s.'3 


B. The Net Costs of JOBSTART 


As presented earlier in this report, 56 percent of control group members received remedial or 
occupational instruction in a program other than JOBSTART, while 94 percent of experimental 
group members received JOBSTART and non-JOBSTART services. The program's impacts are 
therefore the incremental impacts of the services received by experimentals over the mix of services 
available to these youths outside of JOBSTART (represented by the services received by controls). 
For the calculation of a net program cost estimate, the costs of these control services must be 


Lad 


Sin this analysis, some support services are treated as transfer payments to participants and some as 
program expenses that did not affect participants’ income directly. 
Only the Atlanta Job Corps, EGOS in Denver, the Phoenix Job Corps, and the Los Angeles Job Corps 
Offered these benefits. 
overall average cost estimate is a weighted mean of the average costs by site. The weighting was 
done by multiplying each site’s cost per experimental by the number of experimentals in the site and dividing 
the result by 988. 
*The mean of $117 includes values of zero for the sites where these benefits were not offered. 
10S, Department of Education, National Center for Education Statistics, 1989. 
The average dropout grade at baseline was 10 for experimentals in JOBSTART: that is, the average 
JOBSTART participant lost two years of high school. 
12.5, Congress, House Ways and Means Committee, 1992, p. 1692. 
13S, General Accounting Office, 1986. 
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subtracted from the gross program custs presented earlier. However, experimentals also received 
non-JOBSTART services, which they usually pursued on their own after leaving JOBSTART but 
during the 48-month follov'-up period. The costs of these services should also be included in the 
net cost estimate. 

Table 7.4 shows that 94 percent of all experimentals in the 48-month impact sample received 
education or training from JOBSTART or non-JOBSTART sources, compared to 56 percent of all 
controls, for a difference of 38 percentage points. The table also shows that almost all of the 
experimental-control difference in service receipt is accounted for by the JOBSTART services 
received by the experimentals: 363 hours of the 367-hour difference in average service receipt were 
JOBSTART education and training services received by experimentals. This results from the fact 
that experimentais and controls received approximately equal average hours of education and 
training from non-JOBSTART sources (437 hours for experimentals versus 432 for controls).!4 
Therefore, the difference between experimentals and controls in service receipt is approximately 
equal to the amount of JOBSTART services received by experimentals. 

Since the non-JOBSTART services for experimentals and controls were approximately equal, 
the analysis in this chapter assumes that the net cost of providing education and training services to 
the experimentals is simply the cost of providing JOBSTART services. In other words, in this 
special circumstance, the gross cost of providing JOBSTART services is also the net cost of the 
program. 


Ill. The Participant Perspective 


From the perspective of the participant, the effect of a program such as JOBSTART is the 
effect of all financial gains and losses combined, regardless of the source of these gains and losses. 
In other words, income from employment is treated in the same manner as income from public 
sources. Therefore, the participant perspective offers a measure of the net effect of any program- 
induced earnings gains on the participant's disposable income because welfare losses are taken into 
account. 

As shown in Chapter 5, JOBSTART slightly increased participants’ earnings over the four-year 
follow-up period. This increase is not significant for the full sample or for any of the three key 


1The probability of experimentals and controls receiving non-JOBSTART training services differed by only 


1.7 percentage points (39.7 percent for experimentals versus 38 percent for controls), and although controls 
were more likely to receive educational services from non-JOBSTART providers, this did not lead to a 
difference in total hours of services received from non-JOBSTART providers. 
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TABLE 7.4 
IMPACTS ON RECEIPT OF EDUCATION OR TRAINING 


THROUGH MONTH 48, 
BY TYPE OF SERVICE PROVIDER 


Outcome and Service Provider Experimentals Controls 


Ever received education or training 
from any provider* (%) ‘ 56.1 


Ever received education from (%) 
JOBSTART 


Other providers 
Ever received training from (%) 
‘ART 


0 
432 
432 


Difference 


subgroups. However, the point estimates for the earnings effects in each of the four follow-up years 
can be used to assess the program's costs and benefits to the participants. As discussed in Chapter 
5, participation in a voluntary program such as JOBSTART is likely to entail some opportunity cost 
for the participants in the earlier part of the follow-up period. Participants in JOBSTART invested 
their time and effort to improve their skills, expecting this investment to pay off in the future. In 
the meantime, they may have given up the opportunity to work and acquire new skills on the job. 

The first test of JOBSTART’s success is whether it was a useful investment for the participants. 
The impacts, presented in Chapter 5, show that the program raised earnings for experimentals above 
those for controls by a statistically insignificant $214 over the full follow-up period. Underlying this 
impact is a time pattern showing losses in the first two years, which were offset by gains in the last 
two. These earnings gains, amounting to $423 and $410 in years three and four, respectively, are 
almost statistically significant (respective p-values of 0.102 and 0.125).15 

In addition to these earnings impacts, program participants experienced additional program 
effects, some of which cannot be readily expressed in dollar terms. Among the effects that can be 
expressed in dollar terms, and that directly affected the participants’ disposable income, are impacts 
on a number of important public benefits. Chapter 6 showed that for the full sample no impacts 
on these benefits were either large or statistically significant: Overall, there was little or no 
difference in the amount of income that experimentals and controls received in the form of AFDC, 
Food Stamps, and General Assistance over the four-year follow-up period. Since the magnitude of 
these welfare impacts is very small and some are positive and some negative, their net effect on the 
program's benefits for participants is very small. However, as suggested by Chapter 6, the effects 
for the subgroups (presented below) are somewhat different. 

As mentioned earlier, other program benefits that must be included in the analysis are support 
services that were made available to participants while they were in the program. Many of these 
support services were aimed at reducing direct costs associated with individuals’ participation in the 
program, most notably transportation and child care expenses. Because the actual amount of such 
costs to participants are not measured explicitly, offsetting benefits (that is, the support services they 
received) will not be counted either. However, some support services did more than compensate 
for direct costs related to participation in the program. The best example of these are the extensive 
health care services that were offered at some sites, which may have constituted a significant benefit 


1STo the extent that earnings gains are accompanied by increased tax liability, such liability is a loss to 
participants. No tax losses were included in this analysis, although it can be assumed that some occurred. 
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to the participants, one that was not available to controls. The average value of this benefit per 


experimental was $117. Since most program participation occurred in the first year after random 
assignment, this benefit will be counted then. Other support services will be considered 
compensation for unmeasured costs of participation and will not be counted as benefits to the 
participant. 

Table 7.5 and Figure 7.1 show a summary of the program's costs and benefits as perceived by 
the participants. For each of the four follow-up years and for the full four-year period, the table 
presents selected impact estimates from Chapters 5 and 6 (except for medical/dental benefits, for 
which the cost estimate from Appendix C is presented). As its bottom line, the table also presents 
the net present value of the four-year impact on each outcome measure, which is adjusted for 
inflation and foregone returns from alternate uses of program resources.'© 

It is important to note that the four-year net present value includes some estimates that are 
not statistically significant. In other words, one cannot be very confident that these impacts would 
differ from zero if the JOBSTART Demonstration were replicated. For this reason, among others, 
the net present value results based on the measured impacts should be viewed as estimates, the 
precision of which is uncertain. Nonetheless, they do represent the best available evidence of 
JOBSTART's observable economic consequences. 

As Table 7.5 shows, the program's net present value to participants is quite small, $254, but 
positive, indicating that the program had begun to break even by the end of the follow-up period. 
Figure 7.1 shows cumulative program effects over the four years of follow-up.!7 The program's 
payoff did not start to materialize until after the third year, but the initial gap between experimentals 
and controls disappeared entirely by the end of the follow-up period. 

The trend underlying this graph points toward a larger future payoff for experimentals, but the 
data on which these impacts are based cannot be used to confirm the continuation of this trend. 
Future results will depend on the extent to which the program's impacts in the latter two years are 
sustained over time. 

Table 7.6 and Figure 7.2 show the program's estimated net present value for participants in the 
three key subgroups. Four years after random assignment, only women who were custodial mothers 


16The rates of inflation for the second, third, and fourth years of follow-up were 3.6, 4.1, and 4 percent, 
respectively (based on the consumer price index from the U.S. Bureau of Labor Statistics). To this, a rate of 
5 percent for foregone returns from alternate uses of program resources was added, bringing the final 
cumalaeve discount percentages for years two through four to 8.6, 9.1, and 9.8 percent, respectively. 


The graph was created by adding together the impacts in each year and subsequently discounting them. 
The discounted figures were then plotted on the graph. 
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TABLE 7.5 


THE PARTICIPANTS’ PERSPECTIVE: 
SELECTED IMPACTS AND ESTIMATED NET PRESENT VALUES 


THROUGH MONTH 48 
Earnings and General Medical/Demal 

Fringe Benefits® AFDC Food Stamps Assistance Benefits” Total 
Follow-Up Period (S) (S) (S) (S) ($) (S$) 
Year 1 -559 63 -45 24 117 -400 
Year 2 -136 24 -42 7 0 -147 
Year 3 474 3 31 6 0 496 
Year 4 459 -11 31 3 0 482 
Years 1-4 240 74 -24 29 117 432 
Net present value 
of 4-year impact 69 74 -34 28 117 254 


SOURCES: MDRC calculations from JOBSTART enrollment form and survey data (impact figures); Appendix C 
(medical/dental benefit figures). 


NOTES: Impact calculations for this table used data for all sample members for whom there were 48 months of 
follow-up survey data, including those with values of zero for outcomes and those who were assigned to JOBSTART but did 
not participate. See Appendix C for the sample used for the medical/dental cost estimates. 
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tinge benefits were calculated at 12 percent of reported earings. 

Berra Ceemeiks Core ceiecabls Dene escerureica voces of Basal CAG? vas ck tcbbn: ev wbleiteee 
Appendix C). Controls were not eligible for them. For accounting purposes these benefits will be assumed to have occurred 
in the first year after random assignment. 
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FIGURE 7.1 


THE PARTICIPANTS’ PERSPECTIVE: 
CUMULATIVE DISCOUNTED IMPACTS 
THROUGH MONTH 48 


Cumulative Impact ($) 
Ps 


-750 


Random 
Assignment Year 


SOURCE AND NOTE: This graph shows the cumulative net present value of the annual impacts in 
Table 7.5. 
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TABLE 7.6 


THE PARTICIPANTS’ PERSPECTIVE: 
SELECTED IMPACTS AND ESTIMATED NET PRESENT VALUES 
THROUGH MONTH 48, BY GENDER AND PARENTAL STATUS 


Women living with own child(ren) < 
Year 1 -161 95 -98 29 117 -18 
Year 2 168 123 -108 43 0 226 
Year 3 367 33 150 123 0 607 
Year 4 325 -148 167 164 0 508 
Years 1-4 700 37 112 359 117 1,325 
Net present value 
of 4-year impact 553 67 57 298 117 1,093 

Women not living with own 

child(ren), including those who 

did not have any 
Year 1 -384 4 -54 -2 117 -319 
Year 2 85 -191 -69 “1 0 -176 
Year 3 470 310 -96 -98 0 34 
Year 4 516 -277 -121 -107 0 11 
Years 1-4 687 -775 -340 -209 117 -520 
Net presem value 
of 4-year impact 488 -646 -292 -168 117 -501 


TABLE 7.6 (continued) 


SOURCES: MDRC calculations from JOBSTART enroliment form and survey data (impact figures); Appendix C 
(medical/dental benefit figures). 


NOTES: Impact calculations for this table used data for all sample members for whom there were 48 months of 
follow-up survey data, including those with values of zero for outcomes and those who were assigned to JOBSTART but did 
not participate. See Appendix C for the sample used for the medical/dental cost estimates. 


four were discounted to 1986 


Appendix C). Controls were not eligible for them. 
in the first year after random assignment. 


FIGURE 7.2 


THE PARTICIPANTS’ PERSPECTIVE: 
CUMULATIVE DISCOUNTED IMPACTS THROUGH MONTH 48, 
BY GENDER AND PARENTAL STATUS 


Custodial Mothers 
—ji— 
Other Women 


All Subgroups 


SOURCES AND NOTE: This graph shows the cumulative net present value of the annual impacts in 
Tables 7.5 and 7.6. 
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at entry into the program experienced a positive net present value, slightly exceeding $1,000. The 
other two subgroups, men and all other women, were not better off after four years; the program 


resulted in a negative net present value for them. The net present value for men was -$270; for all 
other women it was -$501. 

The negative results for males are explained mostly by initial foregone earnings among 
participating experimentals. This opportunity cost of the program amounts to a loss of earnings and 
fringe benefits of more than $1,350 in the first two years after random assignment. Their later gains 
were insufficient to offset this loss. 

The major reason for JOBSTART's unfavorable effects on the net income of other women is 
that JOBSTART reduced the amount of transfer payments they received compared to their control 
group counterparts. This, in turn, appears to reflect JOBSTART's impact on birth rates for this 
group. As shown in Chapter 6, women in the experimental group who were not custodial mothers 
at random assignment were somewhat less likely to become pregnant or give birth during the follow- 
up period than similar women in the control group. This would make them less likely to become 
eligible for or need programs such as AFDC and Food Stamps. To the extent that negative impacts 
on transfer income are explained by lower birth rates, the loss of this income is offset by a reduction 
in the costs of supporting a family. From a financial point of view, such a reduction in the cost of 
having and raising children implies that the reduced transfer payments are not really experienced 
by these women as an economic loss. 

A similar point can be made about the program’s apparent lack of impact on public assistance 
receipt for women who were custodial mothers at random assignment. Chapter 6 showed that for 
this subgroup there was a positive impact on further childbearing. However, many of these births 
appeared to occur among married women, which may explain why no significant impact on welfare 
receipt was recorded for this subgroup, since AFDC, the most common type of public assistance, 
is provided primarily to women who are single parents. 

In conclusion, it appears that for the full 48-month impact sample the program reached the 
break-even point for experimentals at the end of the four-year follow-up period. For custodial 
mothers, this break-even point was reached much earlier, at roughly 18 months after random 
assignment. Consequently, for this group the payoff at the end of the observation period was much 
larger, slightly more than $1,000. Men, who after four years still faced a negative net present value 
of $270, appeared to be nearing the point at which the program would begin to pay off. However, 
the extent of the ultimate payoff for men will probably be smaller than that for custodial mothers, 
since ongoing future benefits will be worth less and less as a result of inflation and discounting to 
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determine a present value. Finally, for women who were not custodial mothers at random 
assignment, the costs of the program appeared to outweigh its benefits. After four years, no 
financial turnaround had yet occurred for these women, who faced a $501 loss despite encouraging 
earnings gains. Looking only at the program’s effect on family income, these women will benefit 
financially from JOBSTART only if earnings impacts continue to grow in the future. But it is 
important to remember that JOBSTART led to smaller households for this subgroup (owing to 
fewer births), so the available income supports fewer people. 

Finally, the reader must be reminded that the costs and benefits addressed in this section are 
limited to easily measurable, financial program effects. There were many other program impacts of 
JOBSTART (presented in Chapters 4 and 6), and to the extent that impacts on outcomes such as 
educational attainment and family status affect other aspects of experimentals’ lives or affect 
earnings beyond the observation period, the program's full impact may not be captured in this 
analysis. 


IV. Other Benefit-Cost Perspectives 
A. The Taxpayer Perspective 


As expected, the program’s benefits to taxpayers in relation to the full 48-month impact sample 
were very limited (see Table 7.7 and Figure 7.3). In the absence of any real welfare savings for the 
full sample, no payoff in reduced government spending occurred to offset the $4,500 investment 
made in each JOBSTART experimental. Table 7.8 and Figure 7.4 show the findings from the 
taxpayer perspective for the three main subgroups. Note that no tax gains were included in this 
analysis, even though it can be arsumed that participants’ earnings gains did result in somewhat 
higher tax payments. 

The lack of substantial welfare savings for the full sample is related to the target group and 
goal of the program. JOBSTART, in contrast to other welfare-to-work programs, was not targeted 
specifically at welfare recipients. Also, welfare receipt rates for men and for women who were not 
custodial mothers at baseline were low. The only welfare savings of some note, for women who 
were not custodial mothers at entry into the program, appear to have resulted more from lower birth 
rates than from increased employment.!® However, it is possible that for this and other subgroups 


18Note that savings in administrative costs, associated with reductions in welfare payments, were not 
included in this analysis. 


-220- 


318 


-17Z2- 


TABLE 7.7 


THE TAXPAYERS’ PERSPECTIVE: 
SELECTED IMPACTS AND ESTIMATED NET PRESENT VALUES 


THROUGH MONTH 48 
Net Cost of General 

Operating JOBSTART® AFDC Food Stamps Assistance Total 
Follow-Up Period (S) (S) ($) (S) ($) 
Year 1 -4,548 63 45 -24 -4,590 
Year 2 0 -24 42 -7 11 
Year 3 0 3 31 6 -22 
Year 4 0 11 31 3 -23 
Years 1-4 4,548 -74 24 -29 -4,624 
Net present value 
of 4-year impact 4,548 -74 34 -28 4,616 


SOURCES: MDRC calculations from JOBSTART enroliment form and survey data (impact figures); Appendix C 
(operating cost figure). 


NOTES: Impact calculations for this table used data for all sample members for whom there were 48 months of 
follow-up survey data, including those with values of zero for outcomes and those who were assigned to JOBSTART but did 
not participate. See Appendix C for the sample used for the operating cost estimate. 

in calculating the estimated net present value, all costs and benefits in years two through four were discounted to 1986 
dollars ut a rate of § percent plus the rate of inflation as measured by the consumer price index. 

*Th,~ ethod of calculating the JOBSTART operating cost is described in Appendix C. This cost includes medical/dental 
benefits prov. :-d to participants. For accounting purposes this cost will be assumed to have occurred in the first year after 
random assigm: >t. 
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FIGURE 7.3 


THE TAXPAYERS’ PERSPECTIVE: 
CUMULATIVE DISCOUNTED IMPACTS 
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SOURCE AND NOTE: This graph shows the cumulative net present value of the annual impacts in 
Table 7.7. 
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TABLE 7.8 


THE TAXPAYERS’ PERSPECTIVE: 
SELECTED IMPACTS AND ESTIMATED NET PRESENT VALUES 
THROUGH MONTH 48, BY GENDER AND PARENTAL STATUS 


Net Cost of General 
Subgroup and Operating JOBSTART®° AFOC Food Stamps Assistance Total 
Follow-Up Period (s) (S) (S) (s) (s) 
Men 
Year 1 4,548 -1 3 44 -4,590 
Year 2 50 -18 13 45 
Year 3 -47 33 33 -47 
Year 4 -52 42 32 -62 
Years 1-4 4,548 -52 -90 34 -4,539 
Net present value 
of 4-year impact 4,548 -35 -74 20 -4,636 
; Women living with own child(ren) 
5 Year 1 -4,548 -95 98 -29 4,574 
Y Year 2 -123 108 43 -58 
Year 3 33 -150 123 -240 
Year 4 148 -167 -164 -183 
Years 1-4 4,548 -37 “112 -359 -4,939 
Net present value 
of 4-year impact 4,548 -67 -57 -298 -4,971 
Women not living with own 
child(ren), including those who 
did not have any 
Year 1 4,548 4 54 2 -4,496 
Year 2 191 69 1 261 
Year 3 310 96 98 504 
Year 4 277 121 107 505 
Years 1-4 4,548 775 340 209 -3,107 
Net present value 
ot 4-year impact 4,548 646 292 168 3,442 
eee LL 
(continued) 
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TABLE 7.8 (continued) 


SOURCES: MDRC calculations from JOBSTART enrollment form and survey data (impact figures); Appendix C 
(operating cost figure). 


NOTES: impact calculations for this table used data for all sample members for whom there were 48 months of 
follow-up survey data, including those with values of zero for outcomes and those who were assigned to JOBSTART but did 
not participate. See Appendix C for the sample used for the operating cost estimate. 

In calculating the estimated net present value, ee ee 
dollars at a rate of 5 percent plus the rate of inflation as measured by the consumer price index. 

*The method of calculating the JOBSTART operating cost is described in Appendix C. This cost includes medical/dental 
benefits provided to participants. For accounting purposes the cost will be assumed to have occurred in the first year after 
random 

baverage length of stay and hours of participation in JOBSTART varied among the three subgroups in this table; 
icine tin the cost estimate must be considered an approximation of the real program expenses for each subgroup. 


FIGURE 7.4 


THE TAXPAYERS’ PERSPECTIVE: 
CUMULATIVE DISCOUNTED IMPACTS THROUGH MONTH 48, 
BY GENDER AND PARENTAL STATUS 
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SOURCES AND NOTE: This graph shows the cumulative net present value of the annual impacts in 
Tables 7.7 and 7.8. 
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additional benefits will accrue over a period of time beyond the follow-up pcriod, as a result of the 
increased educational attainment resulting from participation in the program. 


B. Combining the Participant and Taxpayer Perspectives 


Even when the many uncertainties surrounding this benefit-cost analysis are taken into account, 
it is likely that, after four years, JOBSTART had taken much more in resources for program 
participation and operations than it had produced in benefits to participants, taxpayers, or society 
as awhole. The final figure for society (-$4,286; see Table 7.9) indicates that the marginal gains for 
program participants ($254) do little to offset the large losses to taxpayers ($4,540). Note that this 
figure was obtained simply by summing the final figures for participants and taxpayers in Table 7.9. 

Implicit in this method of calculating a "societal" benefit-cost estimate is the assumption that 
a dollar for participants is equal in value to a dollar for taxpayers, that is, both bottom-line estimates 
are combined with a simple weight of one. It is sometimes argued that this is not a valid approach: 
Since JOBSTART participants had fewer resources than the average taxpayer, the marginal value 
of resources provided them could be higher (although conventional economic theory is very cautious 
in making such assumptions). According to this view, participants’ gains should be weighted more 
heavily than taxpayers’ losses. Such differential weighting would somewhat reduce the overall loss 


to society. However, the bottom line for society is likely to remain negative at four years after 
follow-up. 


V. Nom-Monetary Program Effects 


In Table 7.1, a number of program costs and benefits were listed that were not included in this 
analysis because they cannot be assigned a dollar value. Some of these effects were discussed in 
earlier chapters and deserve to be mentioned here, since they may help to put the benefit-cost 
results into perspective. They include: (1) the value of education beyond that arising from higher 
earnings; (2) the preference for work over welfare; and (3) the cost of foregone leisure time and 
activities. 

The first effect relates to the remedial character of the JOBSTART program. As a program 
targeted at high school dropouts, one of JOBSTART's primary goals was to increase th educational 
level and attainment of program participants, a goal that was achieved: JOBSTART experimentals 
received considerably more education and training than members of the control group, and during 
the follow-up period many succeeded in attaining a GED or high school diploma. For the 
participants, this program effect had begun to pay off in increased earnings and labor force 


TABLE 7.9 


ACTUAL VALUE OF COMPONENTS OF THE JOBSTART BENEFIT-COST ANALYSIS, 
BY ACCOUNTING PERSPECTIVE 


Accounting Perspective 


Program 
Component Pantcipants Taxpayers Society* 
Increased earnings and fringe benefits $69 $0 $69 
Increased tax payments 
Payroll taxes - + ft) 
Income and sales taxes - + 0 
Reduced use of transfer programs 
AFDC payments $74 $74 $o 
Food Stamp payments -$34 $34 $0 
General Assistance payments $28 $28 $0 
Payments from other public programs - + 0 
AFDC administrative costs 0 + + 
Food Stamp administrative costs 0 + + 
Reduced use of community education 
and training programs 0 + + 
Reduced criminal activity and income - + 
JOBSTART operating c7sts $0 -$3,863 -$3,863 
Compensation for program-related expenses $0 -$568 -$568 
Additional support services $117 $117 $0 
Change in family's financial needs owing to 
program effects on childbearing ? is ? 
Value of education not reflected in earnings + + + 
Preference for work over welfare + + + 
Foregone leisure time and activities - fe) < 
Total $254 $4,540 $4,286 


SOURCES: Tables 7.5-7.8. 
NOTES: All costs are in 1986 dollars. 


that have not been assigned a dollar value are shown as a benefit (+), cost (-), 
ahaha oligecrneslun sterols radio their value. 


-227- 


participation by the end of follow-up, even though subsequent effects on welfare receipt proved 
insufficient to offset the program’s cost to taxpayers. On the other hand, both participants and 
taxpayers may likely have benefited in other ways from these positive education impacts. Education 
may increase people’s involvement in the political process, as well as enhance their participation in 
civic life. Attainment of educational credentials, especially a high school degree or GED, can open 
up opportunities for further education, which may not be reflected in participants’ earnings for many 
years after the program has ended. Also, parents’ educational achievement may affect their 
children’s lives and performance in school as well. 

Regarding the second effect listed above, a benefit-cost analysis from the participants’ 
perspective may show a limited payoff from work, owing to concomitant reductions in welfare 
income. However, it is generally believed that former welfare recipients prefer work over welfare, 
even if not all of their efforts translate into higher income.'? Further, taxpayers may also 
experience a non-monetary benefit from welfare savings in addition to their tax savings, since many 
object to providing income transfers to able-bodied individuals and prefer income to be linked to 
employment. 

Finally, an additional program cost must be mentioned. Aside from the measured opportunity 
cost (i.e., reduced earnings during the first two years after random assignment), JOBSTART 
participants also lost valuable nonwork or “leisure” time while they were in the program or 
subsequently employed. Loss of this time and the resulting foregone activities are difficult to 
measure and value correctly; therefore, this loss was not included in the benefit-cost analysis. 
However, one cannot ignore these program costs, especially since what is commonly labeled “leisure 
time” often accommodates such highly valued activities as raising children or taking care of other 
family members. 


Vi. Conclusion 


Table 7.9 presents a summary of the benefit-cost results drawn from tables and figures earlier 
in this chapter. Two facts become apparent from this table: (1) Many program effects were not 
explicitly valued in this analysis and they may have affected the results, and (2) the program’s bottom 
line after four years of follow-up from both the taxpayer and societal perspectives appears to be 
largely negative. This concluding section will use both these observations in summarizing the results 


'9See, for example, Coalition on Human Needs, 1987. 
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of the benefit-cost analysis, as well as put them in a proper perspective. 


As was noted earlier, only the most important and easily measured program effects were 
included in the benefit-cost analysis. Table 7.9 uses symbols (+, —, 0) to indicate the assumed, 
unquantified value of many components of the analysis, rather than presenting actual estimated 
dollar values. In some cases, these nonestimated costs and benefits are directly associated with other 
components that were measured and discussed in the analysis. Such is the case, for instance, with 
taxes (associated with earnings) and reductions in administrative costs (associated with welfare 
savings). Such secondary costs and benefits comprise only a fraction of the value of those with 
which they are associated. They were not included in this benefit-cost analysis because their overall 
effect on the bottom line was so small as not to warrant further investigation into their actual dollar 
value. 

Other costs and benefits that were not measured in dollar terms cannot be dismissed so easily. 
The non-monetary value of education was mentioned earlier as a potentially important, yet 
unmeasured, benefit. It is hard to assess the value of this benefit within the observation period, and 
even harder to estimate its value in the longer run. It is sometimes argued that many of the benefits 
of education and educational attainment accrue to future generations. 

Another benefit that has not been assigned a dollar value is JOBSTART's apparent impact on 
criminal activity. Even though Chapter 6 did not show a sustained impact on this outcome, a 
significant in-program effect on arrests was found (a 2.6 percent reduction in the first year). It is 
difficult to estimate the value of this reduction to all groups involved, especially since the type of 
atrest and the conviction rate were not analyzed. However, a small gain to taxpayers and society 
may have occurred. 

Finally, for some outcomes it is hard to tell whether they are costs or benefits. Earlier, the 
program's effects on childbearing were discussed. It is known that childbearing in JOBSTART's 
target population is positively associated with welfare receipt. However, it is difficult to establish 
the benefit or cost of a new life to mothers, taxpayers, or society. No attempt was made to do so. 
Nevertheless, in interpreting the results of this benefit-cost analysis, the reader should be aware of 
these non-monetary program effects and their possible relationship to the effects that were measured 
and presented in dollar terms. 
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This chapter moves beyond the research findings presented so far to offer suggestions about 
the most promising program and policy responses to the findings. Although the chapter draws 
primarily on the JOBSTART evaluation, it takes into account results from other recent studies as well 
as operational experience from many youth programs. It also incorporates key findings from 
nonexperimental research on JOBSTART conducted by MDRC staff, which is presented in detail in 
a separate technical paper.! 


1. ASummeary of the JORSTART and Related Findings 


Rigorous impact analyses of programs are complicated to conduct, so the list of studies 
providing solid evidence of the effects of youth employment programs is not a long one,” and the 
evidence is somewhat discouraging. A recent U.S. Department of Labor summary of youth research 
concluded, “The available findings suggest that employment training programs have generally had 
small average impacts on youth. Excluding findings from the Job Corps, there is no evidence that 
any of the programs has more than a modest and short-term effect."? Since the release of that 
report, interim findings on out-of-school youths from the National JTPA Study and the four-year 
JOBSTART findings have become available.‘ 

The interim findings from the National JTPA Study provide evidence of the effectiveness of 


1See Cave and Bos, 1993. 

2See Betsey et al., 1985, for a review of the research in this field as of the mid-1980s, and Smith and 
Gambone, 1992, and other papers in the same volume for a more recent review. Smith and Gambone present 
a table on “major sources of evidence regarding youth employment program effects,” which includes results 
from only eight programs. 

smith and Gambone, 1992, p. 49. 

‘Also released in 1992 were findings from longer-term follow-up for the Summer Training and Education 
Program (STEP), a demonstration that spanned two summers and combined paid work experience, life skills 
training, and education for economically disadvantaged 14- to 15-year-olds at risk of dropping out of school. 
Earlier reports had found promising in-program effects on academic skills and knowledge of health and 
contraception practices. Unfortunately, the longer-term follow-up revealed no impacts on school graduation 
rates, birth rates, or a long list of other outcomes. See Walker, 1992, for an overview of the demonstration 
and findings. Although the youths targeted in STEP were somewhat different from those in the other 
programs evaluated, the findings have contributed to the general pessimism about the effectiveness of programs 


serving young people. 


Title ILA programs serving disadvantaged out-of-school youths within a diverse sample of SDAs. 


Based on 18 months of post-random assignment follow-up, the results show that access to JTPA 
increased the rate at which youths completed high school or passed the GED examination, but made 
little difference in the earnings of young women in general or young men without a prior arrest. 
Further, JTPA appears to have led to a reduction in the earnings of young men with a prior arrest. 
The local programs seemed to have had particular difficulty helping the more disadvantaged young 
men in the research sample: those who were school dropouts, received public assistance, or had not 
worked recently.> While firmer conclusions regarding the JTPA programs must await the release 
of longer-term findings based on 30 months of follow-up, these interim findings illustrate the 
challenge of designing effective youth employment programs. 

It might appear that the most likely conclusion from these evaluations is that almost nothing 
works — that is, has enduring effects — for disadvantaged youths. (The Job Corps stands out as the 
sole exception.) In-program effects can be achieved — active participation in services, impacts on 
intermediate outcomes such as GED receipt — but there are no real long-term effects on 
employment, earnings, and welfare receipt once young people leave the program. But while the 
message from the recent research is far from optimistic, the conclusion that no program has enduring 
effects is clearly overly pessimistic. In JOBSTART, there were apparent earnings gains — “apparent” 
because they just missed the usual standards for statistical significance — in the third and fourth years 
of follow-up, well after the end of program services. In fact, the pattern of earnings impacts 
observed in JOBSTART is similar to that expected for such a program: an initial period when 
participants forego earnings (the “opportunity costs" of being in the program), followed by a period 
when they catch up with controls, followed by a period when their earnings exceed those of controls. 
This pattern suggests that investing in education and training — above the level of services received 
by the control group — can lead to increases in earnings over time. 

The problem in JOBSTART concerned the magnitude and duration of these negative and 
positive earning impacts, not the lack of any payoff: The initial losses for some subgroups were too 
large and the post-program payoff too modest, at least during the four-year follow-up available for 
this study. Table 8.1 summarizes earnings impacts by year for the full sample and key subgroups. For 


SSee Bloom et al., 1993, for the details of the findings. In interpreting them, it is important to understand 
that the impacts measured in the National JTPA Study — as is also the case for the JOBSTART 
Demonstration — represent the effect of services received by the experimental group above the level of services 
received by the control group. In the case of the National JTPA Study, however, the service difference was 
relatively modest. 


-231- 


332 


TABLE 8.1 


IMPACTS ON EARNINGS THROUGH YEAR FOUR 
FOR THE FULL SAMPLE AND KEY SUBGROUPS 


Sample Year 1 Year 2 
Size (S) (S) 


1,941 -499°** -121 


1,872** 2,491 
178 -882 
child(ren) 508-144 150 328 290 625 
Not living with 
own child(ren)* 533-343 76 420 461 613 
Reason for leaving 
regular high school 
School-related 925 312 175 726* 502 s‘1,181 
Job-related 197 -1,108 220 254 405 -230 
Other 819 -45*** §=—- 975 147 196 -578 


SOURCE: MDRC calculations from JOBSTART enroliment form and survey data. 


NOTES: Calculations for this table used data for all sample members for whom there were 48 
months of follow-up survey data, including those with values of zero for outcomes and those who were 
assigned to JOBSTART but did not participate. 

A two-tailed t-test was applied to each difference between average experimental and contro! 
group outcomes. Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; * = 
10 percent. 

"includes women who did not have children. 
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the full sample, a $499 earnings loss in the first year was followed by a smaller loss in year two, and 
gains of $423 and $410 in the third and fourth years, respectively. 

Behind these results is a great variety of experience among subgroups. As mentioned in 
Chapter 5, most of the earnings impact estimates for subgroups were not statistically significant; 
however, the impacts offer suggestive evidence about how JOBSTART worked for different types 
of youths in the sampl’ -or some subgroups there were encouraging apparent earnings impacts. 
Among males, the initial losses were much larger than the average for the full sample, but the 
estimated gains in the later years also appear to be somewhat larger. There was an important 
distinction between males who had not been arrested between age 16 and program entry and those 
who had been: Those with a prior arrest had a shorter period of initial earnings losses followed by 
larger earnings gains in years three and four (with the latter being statistically significant), resulting 
in apparent gains for the entire four-year period. Among females, those who were custodial mothers 
at random assignment had very small initial earnings losses followed by small apparent earnings gains. 
All other women in the sample experienced first-year earnings losses of a somewhat larger magnitude 
than did custodial mothers (but still much smaller than those of men), followed by increasing apparent 
earnings gains in years two through four. For both men and women, the reason for dropping out of 
school also appeared to affect earnings impacts. Young people who left school because of school- 
related reasons (for example, they did not like school, earned poor grades, or had behavior problems) 
had larger than average earnings impact estimates in the latter two years of follow-up and for the 
entire four years. Those who left school for job-related reasons (for example, because they preferred 
to work, needed a job, or wanted to learn a trade) had large initial earnings losses that could not be 
compensated for by the earnings gains in later years, which led to a small four-year earnings loss. The 
subgroup who left school for other reasons (in many cases owing to pregnancy or the birth of a child) 
experienced initial earnings losses during the first two years and only small earnings gains in the 
remaining years, resulting in a sizable — though not significant — earnings loss for the entire four- 
year period. Finally, the earnings impacts at one of the JOBSTART sites — CET in San Jose - 
stand out as much stronger and more consistent over time than those at the other 12 local programs 
in the demonstration. 

While participants would obviously have gained more from the program if their initial earnings 
losses were smaller and later payoffs larger, the benefit-cost analysis presented in Chapter 7 illustrates 
why such improvements would also have been important to taxpayers and society as a whole. The 
substantial investment of resources to provide JOBSTART services requires decreases in 
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experimentals’ initial earnings losses and/or increases in their later earnings for the program to be 
cost-effective in a conventional benefit-cost framework. 

Operating JOBSTART within the JTPA context of the late 1980s was a challenge, and the 
resulting funding uncertainties and impediments to serving low-achieving school dropouts in a low- 
cost, placement-driven system no doubt took their toll on the program. As JTPA programs have 
moved closer to the JOBSTART model — most notably through the dropping of the cost 
performance standard, and through the recent amendments that create a separate year-round youth 
title within JTPA and call for more targeting of youths with multiple employment barriers and the 
provision of more intensive services — many of the institutional barriers to operating programs such 
as JOBSTART will lessen. If JOBSTART had operated under these new JTPA regulations, the 
implementation challenges would have been less severe. 

The remainder of this chapter presents ideas based on research findings and operational 
experience about how to change JOBSTART and similar programs to take advantage of this more 
hospitable environment and improve earnings impacts. It offers suggestions that should be relevant 
to both current JTPA programs and others. The discussion is divided into three parts: suggestions 
on targeting program recruitment, options for lessening the initial earnings losses (the opportunity 
cost of participation), and ways of increasing the earnings payoff in later years. 


I. Targeting Program Outreach 


As discussed in the previous section, in JOBSTART there was substantial variation among 
subgroups in the extent of initial foregone earnings and the size of later earnings gains. The 
subgroup findings provide evidence that a program such as JOBSTART — which had the capacity 
to provide intensive education and training — has the best effect if youths with relatively serious 
employment barriers are recruited to participate. The more promising initial employment prospects 
of youths with fewer barriers to employment are likely to produce large initial earnings losses that 
will be hard to compensate for in later years. 

However, two cautions are necessary in interpreting this conclusion. First, programs less 
intensive than JOBSTART may have a different pattern of subgroup impacts. In the National JTPA 
Study, for example, the 18-month earnings impacts for both males and females tended to be better 
for those who were more job-ready. The relatively short-term interventions funded by JTPA in the 
study sites did not result in large opportunity costs, and the services may not have been intensive 
enough to help the hss job-ready clients. Therefore, because different programs may be more or less 
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effective for different subgroups, no single targeting rule can be applied across the board.® 


Second, the nature of the program experience can change if all participants face serious barriers 
to employment; without some youths who are relatively employable, there will be fewer role models 
and success stories to help motivate youths and provide satisfaction for staff. This implies that the 
key lesson to draw from the subgroup findings is that a substantial percentage of participants should 
have serious barriers to employment, but that programs should strive for a mixture of skills levels 
among youths served. As the percentage of harder-to-serve clients increases, program managers 
should more closely monitor the achievement of intermediate and long-term milestones by 
participants and the morale and motivation of the young people and staff. 

The pattern of subgroup impacts found in JOBSTART suggests the following outreach 
strategies for similar programs: 


* Young women. The findings for young women who were not custodial mothers 
when they entered JOBSTART — especially the reductions in their AFDC receipt 
rate and in the amount of AFDC received — suggest that program operators 
should work. to recruit this group. This will require outreach in the community 
rather than reliance on referrals from other social service or public assistance 
agencies, because most of these women are not receiving any type of public 
assistance and few have ties to public or nonprofit agencies. 


* Males with a prior arrest. Programs can establish links to the justice system to aid 
recruiting, and might even start participation in the program before the end of 
incarceration or while young people are on probation. Further, participation might 
be made a condition for early release for those in prison, or the program could 
operate as an alternative to incarceration for first-time offenders. 


* Young people who dropped out for school-related reasons. Recruiting young 
dropouts into an employment program such as JOBSTART may be easiest if the 
target is youths actively seeking a job or training, but the subgroup findings imply 
that programs should seek ways to identify young people who dropped out for 
school-related reasons. It may be possible to cooperate with local school 
counselors and teachers to find ways to inform young people who leave school for 
such reasons about the alternative to the traditional educational system provided 
by programs such as JOBSTART. For example, programs could get lists from 
school counselors of young people who have recently dropped out owing to 
educational difficulties and make contact with them right away. 


“interestingly, in the National JTPA Study, impacts for males were worse for those with a prior arrest, 
the opposite of the JOBSTART findings. It could be that during enroliment in a government-funded 
employment and training program, staff learn whether experimentals have an arrest record and, if so, feel 
obligated to inform potential employers, while controls may not inform employers if they have been arrested. 
And although JOBSTART was operated principally with JTPA funds (that is, government funds), the impact 
differences for men with a prior arrest may be attributable to the fact that the short-term interventions of the 
ITPA study sites were not substantial enough to overcome this barrier to employment, while the JOBSTART 
services were. 
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For some young people, especially males, participating in JOBSTART resulted in substantial 
Opportunity costs in terms of foregone earnings. The lessons on targeting program outreach, 
discussed above, can reduce this problem somewhat, but it will still be present for some youths. 
These initial opportunity costs can undermine efforts to keep young people in the program long 
enough for them to substantially improve their skills. There is nonexperimental evidence (cited later 
in this chapter) that there may be a threshold level of participation above which earnings impacts are 
larger, and operational experience suggests that providing youths with income while participating in 
a program can improve retention.’ But even for those with substantial participation, initial earnings 
losses can sometimes overwhelm earnings gains in the years following program participation, making 
the participants worse off financially, contrary to the goal of the program. 

There is no single obvious way to address the problem of initial earnings losses; all approaches 
pose trade-offs for program designers and operators and have budget implications. The following 
options provide a range of responses beyond the targeting strategies already discussed. The 
possibilities mentioned are not necessarily consistent with one another, but could be appropriate in 
different circumstances depending on funding and operational constraints. The options include: 
providing income during program participation and restructuring the duration and sequence of 
program services. 

A. Linking Education and Training with Paid Work Experience 

If structured properly and offered as part of a program of education and training, paid work 
experience has the potential — not yet carefully tested — to improve later program impacts in 
addition to its obvious value in providing income during program participation.® This argument 
hinges on several hypotheses. First, the young people's experiences on the job can become part of 


One of the attractions for young people in the YouthBuild program is the opportunity for paid work 
experience. This program, discussed in more detail later in the chapter, has shown strong early success in 


®in the Job Corps, paid work experience is an important part of the program, which also includes 

education and training. Research has indicated that work experience alone does not appear to be effective 

for very disadvantaged youths. In the National Supported Work Demonstration, a program of paid work 

experience was tested for several groups, including very disadvantaged young school dropouts. For these 

youths, the program did not increase long-term carnings and was not cost-effective. However, a follow-up 
combined 


encouraging program 
participation, though lack of funding prevented analysis of program impacts. See MDRC Board of Directors, 
1960, for a summary of the National Supported Work Demonstration, and Scharfman, 1981, for a discussion 
of the special variation of supported work combined with education for young dropouts. 
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the program, serving to make the education and training more relevant and to reinforce their learning 
of skills through application in a real-world setting. Much recent literature has pointed out the 
advantages of such contextual or experiential learning.? Second, work experience can help the 
young people to be socialized into the adult won ld of work in a gradual, nonthreatening way and bring 
them into contact with adult role models in a work setting. Third, there is growing evidence that 
young people value opportunities to make a contribution to their community and that such service 
can change the way they see themselves and relate to others, and the way others see them.'° And 
fourth, since many young people become involved in programs because they are seeking a job, paid 
work experience could lead them to stay in the program longer, learning more academic and 
occupational skills. 

The new JTPA amendments and regulations make paid work experience somewhat easier to 
provide. Work experience in the public or nonprofit sector is permitted for youths when it is 
accompanied by “other services designed to increase the basic education and/or occupational skills 
of the participant,” as wouid be the case in programs such as JOBSTART. Further, other income- 
providing activities such as cooperative education placements or “limited internships” may be arranged 
in private, for-profit firms. On-the-job training, which is now more restricted under JTPA, is also an 
option. 

In considering work experience and related options, however, it is important to understand the 
responsibilities that providing such services entails for program operators. Developing a large number 
of placements may be challenging and demanding, especially when the clients are young people with 
low educational attainment and limited work histories. In addition, it is necessary to monitor the 
nature of the work and the young person’s performance on the job to ensure that the employment 
experience is meaningful and linked to classroom training. Such monitoring entails an administrative 
burden when many employers are involved. Finally, in a slow-growing economy — as in recent years 
— wages may have to be partly or even fully subsidized, raising the costs of the service."! 

These factors create incentives to keep the management and supervision of paid work 


*See Berryman and Bailey, 1992. 

Mgeveral innovative programs offer young people a chance to make a contribution to their community 
while learning important skills. Among them are state and local youth conservation corps programs and 
YouthBuild. 

There has been one large-scale program providing work experience to a large number of youths. The 
Youth Incentive Entitlement Pilot Projects (YIEPP), operated between 1978 and 1980 in 7 large and 10 
smaller locales, were a test of a full-scale job entitlement program for economically disadvantaged youths. In 
the demonstration, 76,000 youths were employed in work experience positions and many program 
implementation issues were successfully overcome. See Gueron, 1984. 
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experience in-house within a youth employment agency or in specially created organizations, as in 
Supported Work or YouthBuild. This increases the chance that employers will understand the initial 
skills and needs of the young people and will take seriously their responsibility to make the 
employment a useful learning experience. To avoid reinforcing unproductive work behavior, 
programs and employers must provide young people with a chance to do real work, set clear 
standards on the job, and not accept weak excuses for poor performance. 


This option for lessening the opportunity costs of program participation can be accomplished 
through either of two almost opposite means: concentrating intense participation in a short time 
period or allowing for extended and less intense participation to enable youths to work while 
enrolled. The choice of approach would depend on the type of young people served. 

1. Concentrating program participation in a short period. In JOBSTART, the site with 
the strongest earnings impacts (CET/San Jose) had a relatively snort average length of stay in the 
program, but much higher than average hours of participation per month. While the length of 
participation is probably not the sole reason for CET/San Jose's strong performance, the site's short- 
term, intensive program minimized the time that young people were out of the labor market, which 
reduced the opportunity cost of participating. This option should be distinguished from a call for 
short-term, low-intensity programs; rather, the approach is more similar to a full-immersion program. 

However, there are trade-offs here as well. An intense and relatively short immersion in 
education and training requires that youths participate more hours per day than was typical in 
JOBSTART sites. Also, other activities such as life skills training must be downplayed despite their 
apparent value for some participants. Youths who will not or cannot invest many hours per day in 
participation would be inappropriate for such a program. Such youths may (1) lack the interest or 
ability to focus for extended periods at the point when they enter the program; (2) have other 
responsibilities that prevent full-day program participation (for example, child care or care of another 
household member!2); or (3) need to work to cover their living expenses. The first problem could 
be addressed by making the program sufficiently engaging, the second by providing appropriate 
support services, and the third by offering paid work experience. However, even these special efforts 
would not help some youths make the necessary commitment. 


12in JOBSTART, the CET/San Jose sample included a relatively low percentage of young mothers because 
the site was simultaneously operating the Minority Female Single Parent (MFSP) Demonstration, which 
targeted this group. In that study, CET/San Jose also produced strong earnings impacts. 
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2. Combining work and program participation over an extended period. This approach 
could be appropriate for youths who want or need to work at least part-time? For them, a 
concentrated program of education and training would not be appealing. Services would have to be 
structured to allow participation to vary over time as the mix of education, training, and work shifted. 
Such programs would be making a long-term commitment to their clients and would expect skills and 
employment options to change gradually. In effect, the approach involves recognizing what is often 
a frustration for program operators (the “on and off" nature of many youths’ participation in 
programs) and designing program services to facilitate less intense but extended participation. For 
relatively employable youths, unsubsidized work could be possible, while for those with substantial 
barriers to employment, paid work experience positions would be needed. 

The challenges for program operators using this approach are substantial. It requires flexibility 
to allow for part-time participation, which could involve scheduling some activities at night and 
permitting open entry and exit in activities or individualized curricula. It also requires staying in 
touch with inactive participants so they know that the program will welcome them back when their 
work schedule or personal situation permits them to participate again. These requiren.ents all 
increase the management burden on program staff. 

There is also the possibility that involvement in the program would never become intense 
enough to make any real difference in the youths’ skills or attitudes. Further, in part-time programs 
it is more difficult to develop the type of peer support and program cohesiveness that can develop 
in more intensive programs. 

Unfortunately, there is no simple solution to the problem of initial earnings losses. Efforts to 
counteract these losses can increase program costs and — absent substantial increases in the 
magnitude of later earnings gains — can reduce the chance that the program will be cost-effective for 
funders or society. Thus, efforts to combat initial opportunity costs for participants should be coupled 
with efforts to improve later earnings impacts, as discussed below. 


IV. Options for Increasing Long-term Payoffs 
This section discusses six options for raising earnings impacts in the post-program period: (1) 
linking program services more closely to the job market; (2) placing more emphasis on addressing the 


13], might also be appropriate for youths who need an initial period of education to raise reading and 
math skills high enough to participate in training. These youths would probably face serious barriers to 
finding unsubsidized jobs, so paid work experience would be an important service for them. 
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developmental needs of youths; (3) creating various means to help more people complete the 
program; (4) helping more participants receive a GED; (5) strengthening job placement assistance; 
and (6) continuing program services after the initial job placement. 


The JOBSTART guidelines called for sites to provide training in occupations in demand in the 
local job market, but not all sites were able to do so. CET/San Jose, the site with the strongest 
earnings impacts in the study, was the most effective in involving employers in developing the 
program's occupational emphasis and curriculum. Training areas were chosen carefully, based on 
analysis of local labor market needs. The site was also unusual in the extent to which educational 
services were shaped by occupational training needs and provided in an integrated way. 

While the earnings impacts for CET/San Jose suggest that these strategies can be effective, it 
is important to raise a cautionary note because of the experience in another site. Chicago Commons 
was the site closest to CET/San Jose in the extent to which education and training were integrated 
and training was tailored to meet the current demands of the local labor market.’* At the two-year 
point, earnings impacts at Chicago Commons were about the same as at CET/San Jose, but in the 
final two years of follow-up they deteriorated sharply, leading to negative total earnings impacts for 
the four-year period. Two possible explanations are that employer demand for the specialized 
occupations in which the youths were trained declined or that the number of workers in these 
occupations sharply increased and the JOBSTART youths were not prepared to adapt their skills for 
other occupations. 

Paid work experience also has the potential to strengthen the link between education and 
training and the labor market, by providing a workplace context in which young people can apply 
what they are learning in the classroom. Further, it may improve longer-term earnings impacts by 
giving youths access to jobs with the possibility of advancement and further employer-provided 
training; these types of jobs might be hard for young people to find on their own. In the Youth 
Incentive Entitlement Pilot Projects, for example, approximately 20 percent of participants placed in 
private-sector work experience positions were later hired as regular employees in the same 
organization.'© 


14For example, many participants were trained in plastic injection molding techniques. 

'SDespite their similarities, there were many differences between Chicago Commons and CET/San Jose: 
Most importantly, Chicago Commons had not served youths in its training programs prior to JOBSTART, 
while CET had done so for many years. 

16See Ball and Wolfhagen, 1981. 
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B. Addressing the Developmental Needs of Youths 


Many of the young people who participate in employment and training programs have lived in 


relative poverty and isolation for much of their lives. One of the greatest challenges these youths 
face is to overcome the emotional deprivation and psychological distress that result from the many 
housing, child care, financial, personal safety, and other problems they confront each day.'? For 
too many there is not even a knowledgeable and trusted older person to help them cope with these 
problems and move through the normal stages of adolescent development. 

Programs must help youths address these issues and teach them the interpersonal skills 
necessary for life at work and at home.'® These include the ability to communicate with different 
types of people clearly, to work productively in a group, to make plans and carry them out, and to 
handle effectively the unexpected events that inevitably crop up in daily life. 

In JOBSTART, many sites gradually realized the importance of addressing these life skills issues 
directly and increased their emphasis on them. In recent years, teaching materials for life skills 
development have been published and many programs, even within JTPA, now include group sessions 
on these topics. The recent JTPA amendments and regulation revisions recognize achievement of 
life skills competencies as part of program performance standards. 

One approach to help young people learn these interpersonal skills is to emphasize youth 
leadership development: that is, give them serious responsibilities within the program. In the most 
innovative cases, such as YouthBuild programs, young people work together in groups, set the rules 
for the group, plan its activities, and carry out the plans. The governance structure is built around 
the young people. In addition, YouthBuild provides an intense work experience activity — renovating 
housing in the local community — that gives young people the opportunity to learn to handle work- 
related problems in productive ways, demonstrate leadership, and derive the satisfaction and self- 
esteem that comes from helping others.? Proponents of the approach believe that through this 
process young people can develop the skills to address the many personal and situational problems 
they face both within and outside the workplace. 


C. Increasing Completion of Program Activities 
One of the most pressing issues for designers and operators of youth programs is understanding 


1*These issues are reviewed in depth in Smith and Gambone, 1992, and other papers in the same volume. 

18See American Society for Training and Development, 1988, for a discussion of the value placed on these 
skills by employers. 

'Residential Job Corps programs also provide activities to encourage this type of development. 
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the relationship between program participation and program impacts. In other words, is there some 
threshold level of service below which impacts are negative or slightly positive and above which they 
are strongly positive? Despite the importance of this issue, researchers studying youth programs have 
had difficulty providing clear guidance. 

The experimental analysis of JOBSTART's impacts presented in previous chapters does not 
answer this question because the overall effect of JOBSTART on the experimentals who were 
offered the program is the average of impacts for subgroups who received different amounts of 
program services; it is possible that the program's impacts varied substantially by level of participation. 
JOBSTART may have had high opportunity costs in the short run but large payoffs later that accrued 
only to those who stayed in the program long enough to learn the skills needed to achieve the 
payoffs. 

Solving this mystery is a challenging analytical problem.” A forthcoming technical paper uses 
a variety of nonexperimental, econometric methods to gain insight into the relationship between 
program participation and impacts.2 While it does not solve the analytical problems involved in 
such research, the findings from the variety of methods are consistent enough to offer suggestive 
evidence on the issue. 

The most straightforward breakdown of JOBSTART’s impacts by participation in the program 
is to distinguish the experimentals who received services from those who did not. Because 
JOBSTART was a voluntary program, it is reasonable to assume that the 11 percent of experimentals 
who were unserved (“nonparticipants") had impacts that were precisely zero, and that the group of 
program participants produced the entire program effect.2? This being the case, JOBSTART had 
an earnings impact of $241 on program participants over the four years of follow-up (the $214 impact 
per experimental divided by 0.89). For the final two years, the nonparticipation correction transforms 


The basic difficulty is that the amount of participation of a young person in the experimental group is 
& post-random assignment characteristic that is affected by observed and unobserved pre-random assignment 
characteristics as well as experiences in the program. This makes it hard to identify the control group 
counterparts of those who participated either more or less in JOBSTART. 

21§e¢ Cave and Bos, 1993. Among the techniques used by the authors to identify the control group 
counterparts of subgroups of experimentals are instrumental variables, two-stage least squares, and a repeated 


matching process. 

Hence, the impacts for participants (as opposed to all experimentals) are calculated by dividing the 
overall impact for experimentals by the percentage of experimentais who actually participated in program 
services. In JOBSTART, the participant figure is 88.8 percent for the full sample, 88.4 percent for men, 89.9 
percent for women living with their own children at random assignment, and 88.3 percent for all other women. 
See Bloom, 1984, and Cave, 1988. 
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an $833 impact for all experimentals into a $938 impact for program participants. 
The next step is to further disaggregate JOBSTART findings for participants by their level of 
participation. The findings from a breakdown by hours of participation suggest that there was 


a specific level of participation at which the program began to pay off for experimentals; prior to 
reaching this level, participants experienced opportunity costs that exceeded their gains. In years 
three and four of the follow-up period combined, the nonexperimental analysis of impacts by amount 
of participation found a program effect of -$983 for early dropouts (fewer than 166 hours), $840 for 
those who received some JOBSTART services (between 166 and 533 hours), and $3,030 for those 
who substantially completed the program (more than 533 hours).2> When compared to the four- 
year earnings impact of $814 for the full JOBSTART sample from the experimental analysis, it 
appears that those participants who spent many hours in the program reaped much greater than 
average rewards. The nonexperimental analysis also found that the impact for nonparticipants was 
very close to zero ($8), a finding that lends some support to the validity of the analysis since it 
corresponds to the expectation that nonparticipants in a voluntary program will have no impacts. 
While the nonexperimental findings are consistent with the hypothesis that increasing service 
levels are correlated to larger four-year earnings impacts, their policy implications are unclear. First, 
the econometric methods used may not completely eliminate the effects of unobserved characteristics 
that could be linked to participation (such as motivation), and it is possible that these unobserved 
characteristics — rather than level of participation — are producing larger impacts. If this is true, 
working hard to keep everyone in the program longer would not necessarily raise average impacts. 
This analytical problem aside, program operators could respond in different ways to the 
suggestive link between participation levels and impacts, although targeting recruitment efforts to 
those who are likely to participate intensively does not seem to be the answer, since in practice it is 
difficult to identify this group.” Further, such targeting may also result in a trade-off mentioned 


23The relatively high participation rate, combined with an already modest impact, leads to a small 
correction that does not change the overall implications of the experimental JOBSTART findings. Also, 
because none of the nonparticipants received any JOBSTART services, benefit-cost estimates are unaffected 
by this correction. 

+The nonexperimental analysis separated the experimental sample into four groups based on participation 
hours: those who did not participate in JOBSTART, and those in the bottom third (fewer than 166 hours), 
middle third (166 to $33 hours), and top third (more than 533 hours). 

Only this last figure is statistically significant. 

26As program operators and researchers have learned through experience, it is hard to predict 
participation levels. Such factors as changes in housing arrangements, child care needs, and health problems 
can affect program participation; these issues are common among disadvantaged young people, but it is 

(continued...) 
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earlier: If young people who are likely to participate intensively are also likely to have good 


employment prospects without the help of the program, they will probably suffer large initial earnings 
losses as an opportunity cost of their participation in the program. There are, however, some clear 
lessons from the JOBSTART Demonstration about how the structure of the program affected 
participation. Youths in sequential/orokered JOBSTART programs (where education was provided 
in-house and the training by other agencies) had a much lower probability of making the transition 
to training and — as a result — averaged lower total hours of participation than youths in other types 
of sites.2” This could have occurred for several reasons: Youths were deterred from going on to 
training by the prospect of applying to a new agency for admission; the training agencies did not see 
these youths as attractive participants; the youths’ real interest was in the education component and 
they did not have an interest in training; or the transition to training occurred so late in the program 
that the youths were already beginning to “drift off to other activities. Agencies considering 
operating sequential/brokered programs should consider the following suggestions to facilitate the 
transition to training: 


1. Develop agreements between service providers to give referrals from the education 
agency priority for admission to training. 


2. Provide opportunities for participants to explore training options during the 
education phase to build their interest and enthusiasm. 


3. Allow youths the option of an early transition to training so that occupational 
training is coupled with basic skills instruction. 


4. Avoid gaps in scrvice between the end of education activities and the beginning 
of training to facilitate a smooth transition. 


5. Use paid work experience to keep youths in the program and to reinforce the 
relevance of skills learned in basic education to training and the world of work. 


6. Streamline the application process as much as possible at the training agency. 


26(...continued) 
difficult at program entry to know how different participants will be affected by them. The participation 
figures presented in Chapter 3 of this report illustrate that this was true in JOBSTART. 

27 as Chapter 3 reported, only 25 percent of experimentals participated in training in sequential/orokered 
sites, and they averaged only 307 hours, as opposed to 387 hours in concurrent sites and 519 hours in 
sequential/in-house sites. There were some probiems in making the transition from education to training in 
sequential/in-house programs as well because of the longer time commitment required for these programs, but 
the transition problems were much more severe in sequential/orokered sites. 

and other suggestions are discussed in more detail in Auspos et al., 1989, Chapter 10. 
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7. Designate a counselor/coordinator or case manager at the education agency to 
monitor and facilitate the progress of youths in the training phase. 


D. Helping More Participants Receive a GED 

‘As discussed in Chapter 4, one of JOBSTART's central objectives was to help participants 
obtain a high school diploma or GED certificate. It was shown that the program was quite successful 
in this regard: By the end of the second year of follow-up, 36.5 percent of experimentals had received 
a high school diploma or GED, compared to 21.3 percent of the control group, for a difference of 
15.2 percentage points, significant at the one percent level.2” However, the impacts on earnings 
were much more modest, raising questions about whether a GED makes little difference in earnings 
or whether it is valuable but JOBSTART helped too few people to get it. 

Attempts to address these questions encounter analytical problems similar to those discussed 
in the previous section. Again, the basic difficulty in such a nonexperimental analysis is identifying 
the control group counterparts of experimentals who exhibit specific types of behavior after random 
assignment. In the forthcoming technical paper mentioned earlier, various techniques are used to 
understand the difference that attaining a GED through JOBSTART made for those who would 
otherwise not have done so.” 

While the nonexperimental findings are again not definitive, a consistent pattern emerged that 
suggests that GED attainment was an important milestone in the JOBSTART program. The 
nonexperimental analysis included an effort to identify members of the experimental group who 
received a GED and their control group counterparts based on characteristics at random assignment, 
regardless of GED receipt during the four-year follow-up period. The earnings impact was especially 
large for the subgroup of these experimentals whose control group counterparts did not receive a 
GED during follow-up. This was in contrast to the small earnings impact for the subgroup whose 
control group counterparts had also received a GED. Thus, the overall JOBSTART earnings impact 
appears to be largely the result of substantial impacts for those youths identified through the 
nonexperimental analysis for whom access to JOBSTART was the key to receiving a GED. 

These nonexperimental findings suggest that JOBSTART’s earnings impacts may have been 
dependent on who was served. It appears that JOBSTART raised the earnings of those who needed 
the program to attain a GED; those who would have reached this milestone on their own or through 


2 Most of this impact occurred through increased GED receipt rather than completion of high school. 

This nonexperimental analysis used the same techniques as the nonexperimental analysis of program 
participation; the control group counterparts were determined through various techniques based on pre- 
random assignment characteristics. 
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other programs, or who would not have reached it in any case, showed small earnings impacts and 
do not appear to have benefited from their JOBSTART experience. In essence, JOBSTART's 
modest earnings impacts may well have resulted because GED impacts were not large enough. If this 


finding is correct, it has targeting implications similar to those discussed earlier in this chapter: Make 
sure that the program includes young people who are likely to be helped to get a GED through it, 
that is, those whose basic skills at program entry are neither so high that passing the GED is likely 
in any case, nor so low that the level of services provided would not be enough to help them pass. 
Although it may be difficult for program operators to identify this “middle” group with precision, basic 
skills testing at the time of application to the program would be helpful in targeting. 


E. Strengthening Job Placement Assistance 

Despite strong job placement assistance at CET/San Jose and a few other sites, it was generally 
the weakest component in the JOBSTART sites. In several sites, the JOBSTART participants were 
served by job placement staff who were also working with more job-ready clients. As a result, the 
understandably busy placement staff tended to work more with the non-JOBSTART clients, who 
tended to be easier to place. This experience highlights the importance of having staff — whether 
job placement staff or occupational trainers — who accept that helping less job-ready clients find 
employment is a central part of their job. 

There were also job placement problems for youths who did not complete the JOBSTART 
services. This was especially apparent in sequential/brokered sites, where a large percentage of young 
people did not make the transition from education to training at another agency. In most programs, 
job placement services were focused on program graduates. It could be counterproductive to provide 
extensive job placement assistance to those in the early stages of program participation, because it 
might induce them to take a low-wage job when, with somewhat more participation, they could find 
a better job. However, it is important to have a "safety net" of placement assistance for those who 
stop participating before completing the program. 

Finally, the JOBSTART experience illustrates the necessity of having job placement staff with 
good connections to employers and the ability to find training-related jobs with long-term prospects 
that are better than those the youths could find on their own. Job placement cannot be an 
afterthought, with responsibilities assigned to staff without the appropriate skills. The demands on 
job placement staff are as challenging as those facing education and training staff. 
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F. Continuing Services After the Initial Job Placement 


Many young people served in JOBSTART found a job after leaving the program but fairly soon 
thereafter lost or left it. Like JOBSTART, many programs — especially those funded under JTPA 
— "terminate" a person at the time of initial job placement in order to claim a positive outcome for 
performance standards. Once terminated, a person cannot receive further services without 
reestablishing income eligibility. This practice seems ill-conceived for two reasons. 

For many program participants, new problems and stresses emerge after they are placed in a 
job and start working. For example, young women with children discover that child care 
arrangements are less reliable than expected. Individuals receiving public assistance learn how tightly 
they may have to budget their resources when they start working and their welfare check is cut. 
Conflicts with fellow workers or supervisors may arise. At the very time when new and serious 
difficulties are appearing, support services are withdrawn. 

Furthermore, this practice does not reflect the fact that few economically disadvantaged young 
people are able to make a major leap in economic status through a first job. More typically, the 
initial job is not particularly good, but it allows the youth to learn new skills on the job and build a 
work history and develop references, which can lead to a better job. 

Some programs have sought to maintain a connection and provide counseling and other 
assistance as clients encounter difficulties in their first job or as they are ready to enter further 
training or find a new job. These programs hold out to participants an open-ended offer of assistance 
in making the many transitions needed to achieve self-sufficiency. They recognize that youth 
development is not a quick or straightforward process; young people try different options, move in 
and out of training and work, and encounter new problems with each new situation. As shown by 
the experiences of both the JOBSTART experimentals and controls, disadvantaged young people are 
not inactive. Most are involved in various education or training activities, or are working. The 
challenge is to help them build on each experience toward self-sufficiency. 

While none of these operational suggestions comes with a guarantee, together they provide a 
strategy for building on the JOBSTART findings to improve the lives of economically disadvantaged 
young school dropouts. 
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APPENDIX A 
DATA SOURCES FOR THE EVALUATION 


Many data sources were used in the evaluation of the JOBSTART Demonstration. 
Baseline demographic data were collected at the time of random assignment. Management 
information system (MIS) data from the sites were used to measure participation hours. Twelve- 
month, 24-month, and 48-month follow-up surveys of sample members were conducted to 
measure impacts on experimentals (including those who did not participate) compared to controls; 
the impacts concerned amounts of education and training received, employment and earnings, and 
other outcomes. The 12-month survey also dealt with the experiences of participants in the 
JOBSTART program. Much qualitative information, including interviews with program staff as 
well as focus groups and in-depth interviews with participants, was used in conjunction with the 
quantitative information. Each data source is described below. 


I. JOBSTART Enrollment Forms 

The JOBSTART enrollment form, designed by MDRC and filled out by program staff at 
the time of random assignment, was the major source of information about the demographic and 
socioeconomic characteristics of sample members. It included data on age, gender, ethnicity, 
family composition, educational attainment, and amount of time since dropping out of school, as 
well as basic information on welfare and employment histories. The enrollment form was com- 
pleted for all but one sample member.! 


Tl. JOBSTART Management Information System Forms 
Sites used a number of MDRC-designed forms to report on the progress of participants in 
JOBSTART. The most important of these are described below. 


1 This sample member was excluded from the impact analysis, since all demographic variables from the 
enrollment form were missing. For many of the sample members, a few specific pieces of demographic 
information were missing. In the impact analysis, the predicted values based on similar sample members were 
substituted for these missing observations. 
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The monthly participation report provided the number of hours that participants spent in 
basic education, occupational skills training, and other kinds of JOBSTART activities each month. 
It also provided information on the type of occupational skills training in which participants in 
training enrolled. Sites reported actual hours attended, not the number of hours scheduled. 

Participation data used in this report were collected from August 1985 — the beginning of 
random assignment — through January 1989. The month of random assignment was included as 
a month of follow-up for participation, although the participant may have been randomly assigned 
late in the month. Those assigned in the last month of random assignment — November 1987 
— had 15 months of follow-up participation data. The vast majority of the sample had at least 
24 months of follow-up. 

Collecting strictly comparable data across sites was not always possible for two reasons: 
First, the services provided in each site varied; second, there was some inconsistency in the way 
sites reported hours for activities other than basic education or occupational training classes. For 
example, a number of sites supplemented education and/or training classes with formal classroom 
instruction in a variety of topics generally termed "life skills." Some sites reported these as 
education hours; others counted them as training hours. In order to have similar definitions of 
the basic components — education and training - MDRC modified the reported hours at sites 
so that time spent in such activities as life skills classes was counted under “other activities." 
The education hours reported by CET/San Jose also were en to reflect only hours spent 
in the site’s GED class.> 


2The sites affected were El Centro in Dallas, the Los Angeles Job Corps, the Atlanta Job Corps, and 
Allentown in Buffalo. At E! Centro, one-half of all education hours prior to December 1986 were spent in 
life skills. After 1986, one-fourth of the reported education hours were spent in life skills. The hours were 
counted as hours in other activities by MDRC. At the Los Angeles Job Corps, participants spent one-half 
of their reported education hours in activities such as art, gym, and “world of work" for the first three months 
after enroliment. MDRC moved one-half of the education hours to hours in other activities for those 
months. At the Atlanta Job Corps, 10 hours each week were spent in activities such as life skills, driver 
education, and health, MDRC moved 28.6 percent of the reported education hours to hours in other 
activities. Allentown in Buffalo included such hours in its reported occupational training hours. MDRC 
moved all reported occupational training hours that did not have an associated type of training to hours in 
other activities. 

3CET/San Jose reported 30 percent of each participant's occupational training hours as education, which 
included time spent on training-related basic skills in occupational training courses as well as hours in the 
site's GED class. For consistency with other sites, the education and training hours at CET/San Jose were 
recalculated by MDRC, and only hours spent in the separate GED class were included as education hours 


in this report. 
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Other differences remained, however. A number of sites offered limited amounts of work 
experience as part of the JOBSTART program. Some sites reported these hours as training 
hours; others reported them under other activities.4 No adjustments were made to these hours. 
Finally, the Phoenix Job Corps did not report hours spent by participants in life skills or 
avocational activities, although the other two Job Corps sites did. 

Appendix Table A.1 shows the common elements and variations in component activities 
across sites. In general, participation hours reported as being in the education component 
consisted of time spent in classes devoted to basic education or GED preparation; they did not 
include work on training-related basic skills done in occupational training courses. In all sites, 
participation hours that were counted in the training component included all activities offered 
in occupational training curricula, including units on training-related educational skills (such as 
Business English or Business Math) and employability development (instruction in work behaviors 
and job search). In the following sites, the hours counted as training also included time spent 
in work experience or on-the-job training: Connelley in Pittsburgh, E! Centro in Dallas, the 
Phoenix Job Corps, and the Los Angeles Job Corps. Hours spent in other activities varied 
considerably across sites and included instruction in life skills, work experience, and orientation 
and avocational activities. 

In order to assess the quality and completeness of the participation data, MDRC staff 
reviewed the teachers’ class attendance records and other sources of data for a randoraly selected 
sample of participants. For the most part, there was agreement between hours found in teachers’ 
records and the monthly participation reports. If more than 20 percent of the cases in a quality 
control sample had discrepancies greater than 10 percent between site-reported hours and hours 
obtained in the check, MDRC scheduled either a recollection of the data or retraining of site 
staff, depending on the seriousness of the discrepancies.° 


4at EGOS {:) Denver, hours spent by participants in ‘work/study" were not reported. 
use it was necessary to obtain records from a number of service providers, many of which did not 
maintain complete records for long periods, occupational training hours in brokered sites were the most 
difficult to confirm and probably have the greatest variation between actual and reported hours. The 
difficulty MDRC staff had in obtaining and verifying data from training providers reflects the difficulty sites 
had in monitoring hours for participants once they were no longer in the site. Problems were found even 
in the two sites with the best data from service providers: One site apparently overreported hours, while one 
site apparently underreported them. Because the number of participants who entered training in 
sequential/brokered sites was small, the misreporting of training hours did not greatly affect the average hours 
of training presented in the report. 
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TABLE A.1 


ACTIVITIES INCLUDED IN PARTICIPATION HOURS, 
BY SITE AND COMPONENT 


Sie Education Training Other Activities 
All shes Classes in basic Classroom occupational Varied 
education or GED skilis training, including 
preparation Classes in training-related 
basic skilis and employa- 
Exceptions, by site 
Allentown (Buffalo) -* va Life skills instruction 
Atiama Job Corps ~e va 10-day orientation, work 
and 
training, fe skills end 
evocational 
included® 
BSA (NYC) A few hours per week in va Life skilis instruction 
computer tite 
akilis instruction may be 
included 
CET/San Jose -_* _* No other activities 
Chicago Commons va wa No other activities 
Connetiey na Work experience No other activities” 
(Pateburgh) mentorships included 
CREC (Hartford) Some hours in wa Work experience 
Gevelop- 
ment activities included 
East LA Skilis wa va No other activities 
EGOS (Denver) va -° No other activites 
El Centro (Dallas) _* Work experience Life skits instruction 
imemships included 
LA Job Cerps -* Work experience and on- 5-day orientation, ilfe skills 
the-job training included and evocational 
Phoenix Job Corps Wa Work experience and on- 8-day 
the-job training included se De aaa 
SER/Corpus Christi wa va No other activities 


SOURCE: Adapted from Auspos et ai., 1969. 


“Reported hours were adjusted by MDAC. 


did not report participation in @ one-hour afterechoo! component consisting of counseling and other support 
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B. Other Management Information System Data 


As part of the monthly monitoring system, sites also reported on the end-of-month status 


of cach participant; the participants who had been terminated and the reason for termination; 
and job placement and GED receipt among participants. The follow-up surveys proved to be a 
more complete source of data for employment and GED receipt, since they included activity by 
experimentals that might not have been reported to site operators as well as the experiences of 
the control group. Consequently, the surveys were the only source of these data used in this 
report. 


lil. Test of Adult Basic Education 


The Test of Adult Basic Education (TABE), a modification of the California Achievement 
Test, was used to measure reading levels of experimentals. Prior research has shown the test to 
be a reliable and valid measure of reading ability. The test was used at two points in time: shortly 
after random assignment (as a baseline measure)® and after participants had spent some time 
in the program (usually after about 100 hours of education), as a measure of reading-level 
gains.’ 
About 20 percent of the total experimental sample did not take a baseline TABE. The 
percentage tested varied by site from a high of 100 percent to a low of 42 percent. The Job 


Corps sites and CET/San Jose had the lowest percentage of experimentals with baseline TABEs. 


IV. Follow-Up Surveys 

Eighty-four percent (1,941) of the 2,311 sample members randomly assigned to the 
experimental or control group between August 1985 and November 1987 were interviewed four 
years after random assignment. (Table A2 presents 48-month survey response rates by site.) 
These 1,941 youths constituted the sample for this report, and each of them provided follow-up 


*In five sites, the TABE was also used as a test of reading-level eligibility and consequently was 
administered to controls as well as to experimentals. A number of other reading tests were administered in 
the other sites. Data from these sites were not included in the analysis because they were not comparable 
across sites. Scores on the eligibility test were used as the baseline measure for experimentals in sites where 
the TABE was used. 

7The actual number of hours of education between random assignment and the first follow-up test varied 
considerably because of differences in measuring hours of education and delays in administering the tests. 
Also, in the first few months of the demonstration, sites were asked to test every three months, which 
resulted in considerable variation in the number of hours after which participants were tested. 
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TABLE A.2 


RESPONSE RATES FOR THE 48-MONTH SURVEY, BY SITE 


Responded Did Not Respond 
Site Sample Size (%) (%) 
Concurrent 
Atlanta Job Corps 80 86.3 13.8 
CET/San Jose 200 83.5 16.5 
Chicago Commons 93 80.7 19.4 
(Pitsburgh) 219 84.0 16.0 
East LA Skills Center 126 84.1 15.9 
EGOS (Denver) 237 83.5 16.5 
Phoenix Job Corps 153 87.6 12.4 
SER/Corpus Christi 300 82.3 17.7 
Sequential/in-house 
El Centro (Dallas) 200 89.5 10.5 
LA Job Corps 296 78.0 22.0 
Sequential/brokered 
Allentown (Buffalo) 147 91.8 8.2 
BSA (NYC) 151 77.5 22.5 
CREC (Hartford) 109 90.8 9.2 


SOURCE: MDRC calculations from JOBSTART enroliment form and survey data. 
NOTES: Calculations for this table used data for all 2,311 sample members randomly assigned in 


JOBSTART. 
Rows may not total 100.0 percent because of rounding. 
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information for 48 months after the date of his or her random assignment. Most responded to 
the 12-, 24-, and 48-month follow-up surveys, while the remainder responded to special 
combination surveys covering the entire period since their last response. The surveys were 
conducted® either in person or, for those who had moved out of the area, by telephone, one, 
two, and four years after random assignment. The interviews lasted about 45 minutes and 


provided information about the sample member's experiences during the period of follow-up 
covered in that survey wave. Respondents were asked about their employment history, family 
status, welfare receipt, and receipt of education or training outside of JOBSTART. For the 12- 
month survey, experimentals who did not participate in JOBSTART were asked why; participants 
were asked what they liked and disliked about the program and their reasons for leaving. 
(Appendix B discusses issues of sample bias and data quality for the surveys.) 

Sample members who could be located were generally willing to be interviewed. Some 
could not be located while others simply could not be contacted. 

Some completed surveys lacked some information that was important in calculating impacts. 
Because the presence of missing data might have been correlated with an observed or unobserved 
prior attribute, dropping cases with missing data from the analysis might have biased the impact 
estimates or produced month-to-month inconsistencies. Imputing values is possible using a 
procedure that does not bias results. A separate regression was run for each variable with missing 
values, yielding predicted values for the missing data. These predicted values were used as 
estimates of the missing values. Continuous outcomes may contain outliers — extreme values that 
overly influence estimates. In the analysis, these were treated as missing, and the usual 
procedures for missing values were applied. 


V. Qualitative Date 
Qualitative descriptions of the program and participants’ experiences in it were obtained 
from a variety of sources and were used to complement the analysis of the quantitative data. 
MDRC research staff visited sites and conducted structured interviews wth program 
administrators, counselor/coordinators, and teaching staff to determine recruitment practices; the 
content of services in the education and training components, job placement, and other activities; 


8MDRC contracted with the survey division of Abt Associates Inc., a Boston-based research firm, to 
implement, manage, and monitor the survey. Completed surveys were data-entered and checked for 
completeness by Abt. Members of the Abt staff also assisted in the design of the survey instrument. 
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the range of support services and retention strategies; and staffing patterns and staff experience 
with JOBSTART. MDRC staff also observed education and training classes in each site and 
visited some of the organizations that provided occupational training to JOBSTART participants 
in the sequential/brokered sites. Sites were typically visited by research staff once during the 
early phase of the demonstration and twice in the second year of program operations. This 
information was supplemented by ongoing reports on program operations and classroom 
observations provided by MDRC operations staff, who visited each site at regular intervals: once 
every month in year one and once every two months in year two of the operational period. 
(Interviews and observations concerning the education component were developed in conjunction 
with an education expert who worked with MDRC as a consultant.) 

Information about participants’ reactions to JOBSTART was obtained from focus group 
discussions with 46 JOBSTART participants in four sites between May 1987 and February 1988. 
Female participants were interviewed at Connelley in Pittsburgh and at BSA in New York City; 
men were interviewed at El Centro in Dallas and at the Los Angeles Job Corps. Each session 
was attended by between 9 and 14 participants and lasted between two and two and a half hours. 
At Connelley and El Centro, the groups were made up of participants in attendance on the 
session day; at the Los Angeles Job Corps, staff selected students who were doing well in the 
program; the BSA group included both current participants in education and women who had 
already moved on to occupational skills training. Because they included many participants who 
stayed longer than average and/or were doing well in the program, the groups were not 
representative of all JOBSTART participants. Nevertheless, used in conjunction with the survey 
responses, the focus group discussions provided valuable insights into participants’ expectations 
about the program, what helped and hindered their participation, their opinions of the education 
and training components, and their recommendations for improving the program. MDRC hired 
consultants to develop the discussion topics, moderate the groups, and analyze the responses. 

A series of in-depth interviews was conducted by another consultant with 15 JOBSTART 
participants in four other sites (CREC in Hartford, EGOS in Denver, Allentown in Buffalo, and 
the Atlanta Job Corps) between November 1986 and September 1987. These profiles provided 
additional, although impressionistic, information about the lives of some JOBSTART participants 
prior to and during the demonstration. The report also drew on the observations of JOBSTART 
staff and selected participants who attended a conference on youth employment initiatives, 
sponsored by MDRC, in October 1987.” 


*See Manpower Demonstration Research Corporation, 1988, for a summary of the conference discussions. 
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APPENDIX B 
THE JOBSTART IMPACT ANALYSIS: METHODOLOGICAL ISSUES 


As outlined in Chapter 2, several methodological issues had to be addressed to answer the 
key evaluation questions with accuracy. 


I. Selection Bias 


Did random assignment succeed in creating a group of JOBSTART controls with the same 
pre-program characteristics as JOBSTART experimentals? If sample members become 
experimentals or controls completely at random, there are no systematic measured or unmeasured 
differences between the two groups before program treatment. Under those circumstances, 
average Outcomes among controls measure what average outcomes would have been among 
experimentals had the treatment not been available to them, and the difference in average 
outcomes between experimentals and controls measures the program's effect. If there are 
systematic preexisting differences between experimentals and controls, then measured differences 
in post-treatment outcomes confound true program effects with biases due to the selection of 
more people from some groups to be experimentals and more people from other groups to be 
controls. 

Table B.1 presents, one at a time, average characteristics for experimentals, controls, and 
both groups together (the full 48-month impact sample). There were only slight differences 
between groups in a few characteristics, and no overall pattern of systematic differences between 
groups. 

An alternative, more rigorous way to deal with this issue is to use linear regression analysis. 
To implement statistical tests for systematic experimental-control differences in those 
characteristics used in impact regressions, Table B.2 presents linear regression results measuring 
the extent of selection bias for the 1,941 members of the JOBSTART sample for whom there 
are 48 months of follow-up survey data. The “full sample" column of Table B.2 shows the same 
slight differences in individual characteristics and the same absence of systematic differences as 
Table B.1. The final entry in the column, the p-value of the F-statistic, is very close to one, 
providing strong evidence that there was no overall pattern of differences between experimentals 
and controls. It shows that random assignment created two groups without systematic overall 
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TABLE B.1 
CHARACTERISTICS AT THE TIME OF RANDOM ASSIGNMENT, BY RESEARCH STATUS 


C u i 
Gender 
Women 1,041 54.7 52.6 53.6 0.357 
Men 900 45.3 47.4 46.4 
Ethnicity 
White, non-Hispanic 172 83 9.4 8.9 0.499 
Black, non-Hispanic 860 44.5 44.1 44.3 
Hispanic 847 44.4 42.8 43.6 
Other 62 2.7 3.7 3.2 
Ethnicky, by gender 
Women 
White, non-Hispanic 97 5.3 4.7 5.0 0.407 
Black, non-Hispanic 467 24.8 23.3 24.1 
Hispanic 451 23.3 23.2 23.2 
Other 26 1.3 1.4 1.3 
Men 
White, non-Hispanic 7§ 3.0 4.7 3.9 
Black, non-Hispanic 393 19.7 20.8 20.2 
Hispanic 396 21.2 19.6 20.4 
Other 36 1.4 23 1.9 
Parental status 
Women living with own 
child(ren) 
No §33 28.6 26.2 27.5 0.207 
Yes 508 26.0 26.3 26.2 
Men who have own child(ren) 
No 765 38.7 42.3 40.4 
Yes 115 6.7 5.1 5.9 
Employed within past year 
No 914 47.2 47.0 47.1 0.945 
Yes 1,027 52.8 53.0 52.9 
Prior employment, by gender 
Women employed within 
past 
No 583 29.7 30.4 30.0 0.317 
Yes 458 25.0 22.1 23.6 
Men employed within past year 
No 331 17.5 16.6 17.1 
Yes 569 27.8 30.8 29.3 
Sample size BS 
(continued) 
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TABLE B.1 (continued) 


Sample Experimentails Controls Full Sample 
‘% 


Characteristic and Subgroup Size (%) (%) (%) p* 

Left school in grade 11 or 12 
No 1,140 §7.7 §9.8 58.7 0.343 
Yes 801 42.3 40.2 41.3 

Received occupational training 

within past year 
No 1,615 84.3 82.1 83.2 0.184 
Yes 326 15.7 17.9 16.8 
16-19 1,425 73.3 73.6 73.4 0.890 
20 or 21 516 26.7 26.4 26.6 

Marital status 
Ever married 184 9.8 9.1 9.5 0.605 
Never married 1,757 90.2 90.9 90.5 

Living in own household or 

with boy/girlfriend 
No 1,575 81.8 80.5 81.1 0.465 
Yes 366 18.2 19.5 18.9 

Own AFDC case or receiving 

General 
No 1,418 74.5 71.6 73.1 0.146 
Yes §23 25.5 28.4 26.9 

Own AFDC case 
No 1,522 79.8 77.0 78.4 0.143 
Yes 419 20.2 23.0 21.6 

Receiving Food Stamps 
No 1,214 63.1 62.0 62.5 0.636 
Yes 727 36.9 38.0 37.5 

Arrested since age 16 
No 1,649 84.7 85.2 85.0 0.764 
Yes 292 15.3 14.8 15.0 

Lived with both parents at 

age 14 
No 1,264 66.5 63.7 65.1 0.195 
Yes 677 33.5 36.3 34.9 

size 1,941 988 953 
(continued) 
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TABLE B.1 (continued) 


Sample Experimentals Controls Full Sample 
Characteristic and Subgroup _Size_(%)_(%) fH) 
Site 
Concurrent 
Atlanta Job Corps 69 3.3 3.8 3.6 1.000 
CET/San Jose 167 8.5 8.7 8.6 
Chicago Commons 75 41 3.7 3.9 
(Pittsburgh) 184 9.2 9.8 9.5 
East LA Skills Center 106 §.1 5.9 5.5 
EGOS (Denver) 198 10.4 10.0 10.2 
Phoenix Job Corps 134 7.1 6.7 6.9 
SEF/Corpus Christi 247 12.7 12.8 12.7 
Centro (Dallas) 179 9.4 9.0 9.2 
LA Job Corps 231 11.7 12.1 11.9 
Allertown (Buffalo) 135 7.2 6.7 7.0 
(NYC) 117 6.1 6.0 6.0 
CREC (Hartford) 99 5.3 4.9 5.1 
1,941 968 953 


SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 


NOTES: Calculations for this table used data for all 1,941 sample members for whum there 
were 48 months of follow-up survey data. Sample sizes reported may fail short of this number 
because of kKems missing from some sample members 

Distributions may not total 100.0 percent because of 

*The column labeled °p’ is the statistical significance level of the difference in distributions of 
Se ee ee observed proportions in each 
subgroup difier by research status only because of random error. A Pearson chi-square statistic was 
used to test the hypothesis of equal distributions. Statistical significance levels are indicated as *** = 


TABLE B.2 


ESTIMATED REGRESSION COEFFICIENTS FOR THE PROBABILITY OF 
ASSIGNMENT TO THE EXPERIMENTAL GROUP 


Custodial All Other 
or Statistic Full Men Mothers Women 
Constant 0.509°** 0.498*** 0.506*** 0.531*** 
(0.011) (0.017) (0.022) (0.022) 
Site 
Conneliey (Pittsburgh) -0.008 -0.057 0.075 0.020 
(0.059) (0.085) (0.115) (0.127) 
CET/San Jose 0.125 -0.058 0.108 0.005 
(0.054) (0.074) (0.190) (0.104) 
SER/Corpus Christi _ - - aia 
EGOS (Denver) 0.014 -0.015 0.136 0.003 
(0.050) (0.076) (0.094) (0.103) 
Chicago Commons 0.028 0.022 0.216 -0.087 
(0.072) (0.100) (0.156) (0.155) 
El Centro (Dallas) 0.014 0.018 -0.078 0.119 
(0.055) (0.080) (0.105) (0.118) 
BSA (NYC) 0.016 0.052 0.100 0.026 
(0.061) (0.087) (0.134) (0.128) 
Allentown (Buffalo) 0.041 0.008 0.139 0.161 
(0.062) (0.095) (0.117) (0.129) 
CREC (Hartford) 0.019 0.011 0.028 0.047 
(0.064) (0.100) (0.125) (0.122) 
Phoenix Job Corps 0.019 0.103 0.028 0.211* 
(0.057) (0.062) (0.116) (0.118) 
East LA Skilis Center -0.021 -0.062 0.218 -0.076 
(0.060) (0.080) (0.159) (0.120) 
LA Job Corps -0.006 0.099 0.121 0.036 
(0.052) (0.077) (0.103) (0.105) 
Atiamta Job Corps 0.026 0.017 -0.060 0.052 
(0.075) (0.115) (0.138) (0.155) 
Male 0.045 _ os _ 
(0.030) 
(continued) 
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TABLE B.2 (continued) 


Custodial 


All Other 


Regressor or Statistic Full Sample Men Mothers Women 


White, non-Hispanic 


Receiving Medicaid 


Left school in grade 11 or 12 


-0.027 
(0.046) 


0.024 
(0.033) 


-0.057 
(0.075) 


0.009 
(0.028) 


0.052 
(0.055) 


0.075 
(0.082) 
0.001 
(0.037) 


0.013 
(0.064) 


0.048 
(0.041) 


0.078 
(0.061) 


0.028 
(0.039) 
0.024 
(0.033) 
0.010 
(0.043) 
0.075** 
(0.036) 
0.028 
(0.034) 


0.027 
(0.025) 


-0.063 
(0.070) 


0.037 
(0.053) 
0.076 
(0.107) 


0.019 
(0.041) 


0.062 
(0.094) 


0.014 
(0.064) 


0.281 
(0.296) 


0.013 


-0.032 
(0.085) 


0.008 
(0.065) 


0.024 
(0.130) 


(0.100) 


0.018 
(0.106) 


0.241 
(0.189) 


-0.008 
(0.081) 


-0.053 
(0.070) 


0.088 
(0.100) 
0.054 
(0.068) 


0.036 
(0.073) 


0.029 
(0.049) 


(continued) 


TABLE B.2 (continued) 


Custodial All Other 
ressor or Statistic Full Sam; Men Mothers Women 
Lived with both parents 
at age 14 -0.034 0.005 0.044 -0.064 
(0.026) (0.037) (0.054) (0.049) 
Employed within past year -0.006 -0.073 0.035 0.040 
(0.025) (0.038) (0.050) (0.047) 
Number of observations 1,941 900 508 §33 
Number of experimentais 968 448 257 263 
Number of controls 953 452 251 250 
Degrees of freedom for error 1,909 870 479 504 
Error mean square 0.251 0.249 0.252 0.254 
R square 0.011 0.036 0.049 0.037 
Mean of dependent variable 0.509 0.498 0.506 0.531 
F-statistic 0.707 1.108 0.871 0.687 
P-value of F-statistic 0.884 0.318 0.658 0.687 


SOURCE: MDRC calculations from JOBSTART enroliment form and survey data. 


NOTES: The dependent variable in each regression equation was unity for each 
experimental and zero for each control. Each characteristic on the right-hand side of each equation 
was measured as a deviation from its mean. The standard error of each coefficient estimate is 
enclosed in parentheses. 

A two-tailed t-test was applied to each coefficient estimate. Statistical significance levels are 


differences in characteristics before enrollment. There was a statistically significant difference 


in only one individual characteristic for the full sample: Experimentals were more likely to live 
in a household with someone else who received AFDC. 

The procedure used to calculate all the impacts for the full sample presented in this report 
took this slight difference in characteristics into account, and estimated the impacts that would 
have occurred had it not existed. A similar approach was used for the three key subgroups listed 
in the remaining columns of Table B.2 when calculating their impacts. 


Tl. Nomresponse Bias 

Were those sample members for whom there are continuous data for 48 months representa- 
tive of the full JOBSTART sample, including survey nonresponders? A high degree of mobility 
among disadvantaged young dropouts makes it difficult for survey interviewers to locate all of 
them a year or two after they have been enrolled into a research sample. As noted in Appendix 
A, 1,941 of the 2,311 full-sample members furnished data covering 48 months, either at the 12- 
month, 24-month, and 48-month junctures, or at the 48-month juncture, for an overall response 
rate of 84 percent (85 percent for experimentals and 83 percent for controls).! See Table A.2 
for site-specific information on response rates. 

There were systematic differences in characteristics between those who responded to the 
surveys and those who did not respond. Table B.3 presents linear regression results measuring 
the extent to which average characteristics for the 1,941 survey responders differed from average 
characteristics at random assignment for the 370 nonresponders. Since the final entry in the “full 
sample" column, the p-value of the F-statistic, is zero to three decimal places, there is strong 
evidence of systematic differences between responders and nonresponders. Responders were 
significantly less likely to be male, and significantly more likely to be white or Hispanic, to be age 
20 or 21, to have left school in grade 11 or 12, and to have lived with both parents at age 14. 
Better response was found at E! Centro in Dallas, Allentown in Buffalo, and CREC in Hartford, 
even after taking differences in individual characteristics into account. Importantly, responders 
were not significantly more likely to be experimentals than controls. 

When nonresponse is randomly distributed among members of both treatment and control 


1 There are two types of nonresponse. Unit nonresponse is the failure to ascertain answers to any of the 
questionnaire items. Item nonresponse is the failure to obtain only some answers. All the response rates 
discussed here are unit response rates. 
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ESTIMATED REGRESSION COEFFICIENTS FOR THE PROBABILITY OF 


TABLE B.3 


UNIT SURVEY RESPONSE 
Custodial All Other 
Full Sample Men Mothers Women 
or Statistic 1,941/2,311 1,130 ‘578 
Constant 0.840*** 0.796°** 0.8 0.884*** 
(0.008) (0.012) (0.013) (0.013) 
Experimental status 0.017 0.033 0.010 0.027 
(0.015) (0.024) (0.027) (0.026) 
Ske 
Connelley (Pittsburgh) 0.039 0.096 -0.047 0.056 
(0.039) (0.061) (0.066) (0.077) 
CET/San Jose 0.007 0.027 0.130 0.002 
(0.035) (0.052) (0.104) (0.063) 
SER/Corpus Christi an sea a ae 
EGOS (Denver) 0.001 0.004 0.006 0.029 
(0.033) (0.053) (0.056) (0.062) 
Chicago Commons 0.004 0.049 -0.039 -0.056 
(0.047) (0.069) (0.091) (0.092) 
El Centro (Dallas) 0.106*** 0.147** 0.053 0.073 
(0.036) (0.057) (0.063) (0.072) 
BSA (NYC) -0.025 0.007 0.121 -0.123* 
(0.039) (0.060) (0.083) (0.072) 
Allentown (Buffalo) 0.106*** 0.200°* 0.090 0.013 
(0.042) (0.069) (0.070) (0.076) 
CREC (Hartford) 0.096** 0.102 0.083 0.088 
(0.043) (0.072) (0.076) (0.074) 
Phoenix Job Corps 0.050 0.061 0.082 -0.004 
(0.038) (0.059) (0.070) (0.072) 
East LA Skills Center 0.018 0.106* 0.113 0.079 
(0.040) (0.059) (0.090) (0.071) 
LA Job Corps 0.033 0.032 -0.061 -0.076 
(0.034) (0.054) (0.061) (0.063) 
Atlanta Job Corps 0.068 0.106 0.038 0.007 
(0.050) (0.063) (0.063) (0.091) 
Male -0.090°** -_ i a, 
(0.020) 
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ressor or Statistic 
White, non-Hispanic 


Hiapani 

Other ethnicity 
Age 20 or 21 

No phone number on 
enrolimem form 
Male parent 


Female parent living 
with own child(ren) 


Limited English 
Arrested since age 16 
Convicted since age 16 
Own AFDC case 
Receiving Food Stamps 
Never married 
Household AFDC case 
Receiving Medicaid 


Left school in grade 11 or 12 


TABLE B.3 (continued) 


Full Sample Men 

1,941/2.311 1,130 
0.058* 0.036 
(0.031) (0.048) 
0.050** 0.052 
(0.022) (0.037) 
0.072 0.004 
(0.049) (0.076) 
0.044** 0.038 
(0.019) (0.029) 
0.024 0.030 
(0.036) (0.063) 
0.023 0.001 
(0.034) (0.040) 
0.022 - 
(0.025) 
0.060 0.041 
(0.041) (0.068) 
0.005 0.019 
(0.027) (0.034) 
0.061 0.071 
(0.039) (0.048) 
0.026 0.063 
(0.026) (0.057) 
0.005 0.036 
(0.022) (0.037) 
0.017 0.034 
(0.029) (0.057) 
0.016 0.018 
(0.024) (0.039) 
0.017 0.009 
(0.022) (0.039) 
0.096°* 0.066 
(0.016) (0.026) 


TABLE B.3 (continued) 


ressor or Statistic 


Employed within past year 


Number of observations 


Full Sample 
1,941/2.311 


0.047*** 
(0.017) 


0.017 
(0.016) 


2,311 


Custodial All Other 
Mothers Women 
1,130 578 
0.053** 0.052 0.030 
(0.026) (0.032) (0.030) 
0.044* 0.010 0.013 
(0.027) (0.030) (0.028) 
1,130 578 603 
551 296 315 
579 282 288 
1,099 548 573 
0.161 0.102 0.102 
0.010 0.092 0.058 
0.796 0.879 0.884 
1.394 1.919 1.218 
0.078 0.003 0.202 
equation was unity for survey response 


value is to zero, the more important are differences in characteristics between survey responders and 
nonresponders. 
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groups, it is troublesome only because it reduces the sample size and thus the statistical power 
to find impacts of a given magnitude. Randomly distributed nonresponse does not alter the 
expected values of adjusted mean outcomes, and thus does not bias impacts. However, when 
nonresponse is greater among one research group (which is not the case for this sample) or 
among members of either research group with certain characteristics (such as men), impacts may 
be biased slightly unless corrected for nonresponse. The most flexible correction for nonresponse 
is incorporation of an additional equation for survey response into a two-equation system with 
the impact equation. The success of attempts to implement such corrections is data-dependent, 
and the differential response rates found do not seem quite large enough to warrant such 
uncertain measures. 


Til. Impact of Participation Versus Impact of Assignment 

Because the target population for the JOBSTART Demonstration consisted of young 
people w'o had histories of dropping out of education p:ograms, it was difficult to get those 
selected for the program to attend and to retain attendees for substantial periods of time. 
However, everyone assigned to experimental status was included when calculating average impacts 
of JOBSTART. Therefore, impacts do not measure the impacts of participation in JOBSTART, 
but rather of assignment to the group eligible to receive JOBSTART services.2 Thus, impact 
estimates average net outcomes for all experimentals, including nonparticipants. Nonparticipation 
“waters down" the program effect the experiment seeks to detect. Fortunately, only 111 of the 
988 experimentals in the 48-month inpc<ct sample never participated in the program. Such low 
ncnparticipation may have been due in part to successful negotiation with sites to place the point 
of random assignment after initial assessment but immediately before program services started. 

When substantial nonparticipation occurs during an experimental evaluation of a program, 
techniques are available for calculating impacts of participation as well as impacts of assignment. 
When the proportion of assignees to the program who are not counted as participants is an 
unbiased measure of the proportion of controls who would not have participated, when the 


2Some might suggest that nonparticipants be excluded from impact analyses. However, such exclusions 
would expose impacts to possible selection biases, undermining the control group's validity in measuring what 
would have happened without the program. When nonparticipants are excluded from the experimeni:s| 
group, average measured and unmexsured characteristics of experimentals may no longer be the same as 
average control group characteristics. See Cave, 1988. 
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program has no effect on nonparticipants, and when the sample is large enough, it is 
approximately valid to use the formula® 


Impact of assignment 
Impact of participation = 
Fraction participating 

Using this formula necessitates validating all of the assumptions underlying it, and thus 
makes impact analysis more complicated than a simple comparison of average outcomes for those 
assigned to the experimental group and those assigned to the control group. The assumption of 
zero effects on nonparticipants is troublesome, because the process of recruiting experimentals, 
screening them, and contacting them when they do not appear may alter their behavior. Thus, 
in this report, impacts of assignment were reported instead of impacts of participation. 

As outlined above, impacts of assignment to JOBSTART were calculated by comparing 
average outcomes for all those assigned to the experimental group with average outcomes for all 
those assigned to the control group. In order to increase the statistical precision of the impact 
estimate, a variant of simple group averaging known as one-way linear analysis of covariance was 
used for the impact analysis in this report. As shown for the 48-month sample of 1,941 survey 
responders in Table B.4, in a multiple regression of outcome on covariates measured at the time 
of enrollment and on a dummy variable for research status, the coefficient of the dummy variable 
is the impact. This coefficient may be interpreted as the difference between the adjusted mean 
outcome for those assigned to the experimental group and the adjusted mean outcome for those 
assigned to the control group. Adjustment removes the effect of slight differences at the time 
of enrollment in characteristics related to the outcome, and yields a purer measure of the effect 
of research status alone. 

Some of the subgroup results presented in this report are based on slightly more complex 
regression equations, which include terms for interactions between experimental status and 
subgroup characteristics. Such "two-way ANCOVA" impacts may differ to some extent from 
“split-file” impacts estimated by eliminating other subgroups from “one-way ANCOVA" analyses 
for Table B.4. However, calculating two-way ANCOVA impacts easily permits determining the 
statistical significance of impact differences, and is less burdensome computationally. 


3See Cave, 1988; Auspos, Cave, and Long, 1988, Appendix E; Bloom, 1984; and Farkas et al., 1984, p. 
85. If such an adjustment factor were appropriate here, its value would be approximately the reciprocal of 
the rate of participation in JOBSTART, or 1 / (1 - 111/988) = 1.127. 

“See Cave, 1987, and Ostle, 1975. 
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TABLE 6.4 


ESTIMATED REGRESSION COEFFICIENTS FOR SELECTED OUTCOMES 


Dependent Variable 
Ever Received Received GED or 
Any Education High School 
or Training, Diploma by End of EverEmployed, = Total Earnings, 
or Statistic Months 1-48 (% M 48 M 37-48 (%) Months 37-48 
Constant 56.140*** 28.570*** 64.484*** §,182.009*** 
(1.241) (1.498) (1.439) (190.042) 
Experimental status 37.887*** 13.407¢** 1.252 410.427 
(1.744) (2.106) (2.023) (267.115) 
Site 
Conneliey (Pittsburgh) 4.848 3.647 -14.483*** -567,502 
(4.484) (5.414) (5.200) (686.639) 
CET/San Jose -12.677°** -8.979° 6.827 3,314.370°** 
(4.119) (4.972) (4.776) (630.623) 
SER/Corpus Christi _ _ x ase 
EGOS (Denver) 3.800 -16.527°** 2.452 1,436.536** 
(3.801) (4.588) (4.407) (581.950) 
Chicago Commons 2.828 -28.284*** 0.072 1,178,044 
(5.486) (6.623) (6.361) (839.945) 
El Centro (Dallas) -5.761 5.567 5.567 1,393.053** 
(4.171) (5.036) (4.837) (638.698) 
BSA (NYC) -0.368 -5.128 -15,119°** 1,525.701** 
(4.658) (5.624) (5.402) (713.296) 
Allentown (Buffalo) 14.218°** -2.662 -17.892°** -667.935 
(4.751) (5.736) (5.510) (727.489) 
CREC (Hartford) -2.856 -16.671°** -29.055°** 309.861 
(4.846) (5.850) (5.619) (741.975) 
Phoenix Job Corps 7.174" -10.467** -17.704°** 416.423 
(4.318) (5.213) (5.007) (661.121) 
East LA Skills Center 10.109** -25.741*** 3.960 2,704.126*** 
(4.592) (5.544) (5.325) (703.112) 
LA Job Corps 1.963 -12.121** 5.475 2,254.905*** 
(3.953) (4.772) (4.584) (605.232) 
Atlanta Job Corps 11.978** -7.344 6.853 1,335,581 
(5.740) (6.930) (6.656) (878.901) 
Male -4.967°* -1.646 17.290°** 3,199.642*** 
(2.258) (2.726) (2.618) (345.738) 
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or Ti ; Diploma by End of Ever Employed, Total Earnings, 
Regressor or Statistic Months 1-48 (%) Month 48 (%) _ Months 37-48 (%)__ Months 37-48 (S) 
White, non-Hispanic 4.325 7.786* 12.889°** 1,984.234°** 
(3.487) (4.210) (4.044) (533.940) 
Hispanic -2.038 -2.996 8.241%" 1,448.159°** 
(2.552) (3.081) (2.960) (390.789) 
Other ethnicity -1.405 -14.432°* 12.375* 1,877.268°* 
(5.735) (6.924) (6.651) (878.160) 
Age 20 or 21 0.706 3.653 3.634 -497.072 
: (2.132) (2.573) (2.472) (326.382) 
No phone number on 
enroliment form -5.939 -11.822** -2.915 -568.788 
(4.168) (5.031) (4.833) (638.142) 
Male parent 4.171 -5.019 6.856 1,003.750 
(4.005) (4.836) (4.645) (613.302) 
Female parent living 
with own child(ren) -2.655 -1.640 4.130 -6.260 
(2.847) (3.436) (3.301) (435.855) 
Limited English -0.979 5.994 7.402 221.686 
(4.910) (5.927) (5.694) (751.783) 
Arrested since age 16 1.712 -6.288* -1.956 -566.609 
(3.101) (3.744) (3.596) (474.833) 
Convicted since age 16 0.083 5.167 -10.607** -1,931.240°** 
(4.642) (5.604) (5.383) (710.750) 
Own AFDC case 5.868** 4.920 -7.147°* -757.830* 
(2.970) (3.505) (3.444) (454.750) 
Receiving Food Stamps -1.982 2.876 1.691 17.163 
(2.488) (3.003) (2.885) (380.684) 
Never married -0.424 3.139 4.967 -332.938 
(3.279) (3.958) (3.802) (502.059) 
Household AFDC case 2.204 -5.643* 4.544 -966.466** 
(2.756) (3.327) (3.196) (421.994) 
Receiving Medicaid 4.347" -0.387 -0.590 -114.056 
(2.581) (3.116) (2.993) (395.156) 
Left school in grade 11or12 -0.767 7.578°** 3.319 992.220°°* 
(1.683) (2.273) (2.183) (268.267) 
-273- (continued) 
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TABLE 6.4 (continued) 


Dependent Variable 


Ever Received Received GED or 
Any Education High School 
or Training, Diploma by End of Ever Employed, _— Total Earnings, 
ressor or Statistic Months 1-48 (% Month 48 (% Months 37-48 (%) Months 37-48 


3.358 321.694 
(2.257) (297.984) 


9.100*** 1,572.63°** 
(2.186) (288.680) 


1,941 2,311 
988 1,162 
1,149 


1,908 
34,218,685. 1 
0.188 
§,390.923 


0.000 


SOURCE: MDRC calculations from JOBSTART enroliment form, MIS, and survey data. 


NOTES: Ordinary least squares regression coefficients in this table correspond to impact 
estimates presented in Tables 4.1, 4.5, 5.1, and 5.4. A one-way linear analysis of covariance procedure 
was used to control for up to 31 kinds of difference in characteristics before random assignment (see 
Ostie, 1975, p. 461; and Cave, 1987). The standard error of each coefficient estimate is enclosed in 


Each characteristic on the right-hand side of each equation was measured as a deviation from 
its mean. 


A two-tailed t-test was applied to each coefficient estimate. Statistical significance levels are 
indicated as *** = 1 percent; ** = § percent; * = 10 percent. 
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Youths in the impact sample can be grouped based on their individual pre-random 


assignment characteristics or on the characteristics of the sites at which they applied for 
JOBSTART. Because such subgroup comparisons are a central part of the analysis presented 
in this report, it is important to discuss briefly the complications in drawing conclusions from any 
observed differences. Crucial comparisons of this type are between men and women, and 
between types of sites. To summarize, impacts can be compared across subgroups defined by 
individual or site characteristics, but more caution is advised in interpreting such results than in 
interpreting the full sample or within-site impacts described in the previous section. This is 
especially true for comparisons of site types. 

The basic reason for such caution is that since sample members were not assigned randomly 
to these subgroups or types of sites, it may be impossible to attribute the difference in impacts 
to the single characteristic used to define the groups. For example, if women have bigger impacts 
than men, it may not be because they are women; the impact difference might really be because 
the women had less prior work experience, so controls were less likely to be working in the 
follow-up period. Further, using individual sites or site groupings for a subgroup impact 
comparison is fundamentally different from using an individual characteristic such as gender. 
Many things about sites differ (such as the local labor market, participant characteristics and 
interests, and program characteristics), and there is a real danger that impact differences for site 
groupings may be misinterpreted as measuring the relative efficiency of a single feature of a site’s 
program, such as its curricula, facilities, or program structure (that is, brokered versus in-house 
services of concurrent versus sequential education and training), rather than other factors that 
could be driving inter-site variation. 

The internal validity of impact comparisons by individual characteristics is difficult to test. 
However, there is a simple test for internal validity of impact comparisons by program features. 
If groups of individuals randomly assigned at two locations really differed only in the features of 
the programs that experimentals could attend, then the post-random assignment experience of 
controls at the two locations should be identical. This rarely happens; more typically, the 
experience of controls varies between sites, just as that of experimentals does.) 


SThe usual situation is known in evaluation literature as "ecological correlation bias." 
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This problem can also affect impact comparisons for subgroups defined by characteristics 


of individuals as well as subgroup impacts by site or site grouping, although usually the concerns 
about misinterpretation are less severe. For example, if virtually all the Hispanics recruited into 
a demonstration are concentrated in one or two sites, then impacts for Hispanics are really site 
impacts. However, normally (and in the JOBSTART Demonstration, as discussed below) most 
measured characteristics of individuals are distributed fairly evenly across sites.© For example, 
there were younger and older sample members, people reading at higher and lower levels, and 
parents and childless youths in all the JOBSTART sites. Moreover, relevant unmeasured 
characteristics of individuals such as degree of motivation to attend a program and desire for a 
GED are likely to have been distributed fairly evenly among younger versus older sample 
members, those with higher versus lower reading levels, those who were parents versus those who 
were childless, and other subgroups defined by the observed characteristics of individuals.” For 
example, if the impact on educational attainment was higher among low-reading-level than among 
high-reading-level sample members, it is reasonable to interpret this as evidence that the 
programs were more effective with the former subgrcup, rather than concluding that higher 
educational attainment occurred because the average sample member who had a low reading level 
was more motivated or wanted a GED more than the average sample member who had a higher 
reading level. 

In contrast, when a sample is split by site or site group, unmeasured characteristics will be 
distributed unevenly across groups. For example, sites that were known for providing education 
services and that offered a sequence of basic skills instruction followed by occupational training 
at another agency (sequential/brokered sites) may have been more likely to recruit clients 
motivated to get a GED than were concurrent sites with a reputation for training, whose typical 
client may have wanted to learn occupational skills. Thus, a finding that JOBSTART's impact 
on GED attainment was smaller at concurrent sites than at sequential/brokered sites does not 
necessarily mean that someone with average motivation and desire for a GED has a better 
chance of getting a GED in a sequential program. Such a finding could mean that those 


‘Several ethnic groups were concentrated in a few JOBSTART sites, however. 
"To lessen this problem, impact estimates presented in this report always used site dummies as covariates 
when calculating impacts by individual characteristics; these dummies can correct for small differences 
between subgroups defined by individual characteristics in unmeasured characteristics associated with site. 
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recruited at sequential sites had, on average, very different levels of desire for skills training 
relative to GED preparation compared to those recruited at concurrent sites. Even if the usual 
statistical adjustment methods are employed in calculating impacts, little can be done about this 
problem, since motivation to participate in particular components was not measured.® 


Ssubgroup impact equations for groupings by site type cannot use individual site dummies to correct for 
small unobserved differences in groups in the same way that equations for groups defined by individual 
characteristics can: Individual site dummies would be highly correlated with the site groupings. To measure 
which delivery system is better for those with average levels of motivation, desire for GEDs, and other 
unobserved characteristics, the best approach is to randomly assign people to each delivery system in cach 
location after carrying out a common recruitment effort at that location. In that way, unmeasured 
chara-teristics would be the same for each delivery system, because each delivery system would be fairly 
represented in each location and in each recruitment effort. 
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APPENDIX C 


This appendix describes the data sources and methodology used to estimate the cost of the 
JOBSTART program in each of the 13 demonstration sites.' It also discusses the factors 
contributing to the wide variation in costs across sites and examines the relative influence of 
different JOBSTART components on overall program costs. 

The central objective of the analysis was to identify the market value of all resources used in 
providing JOBSTART services. It therefore counted as program costs not only the expenditures 
made by the agencies sponsoring the program, but also those made by outside organizations 
responsible for providing certain components (such as occupational skills training in the three 
sequential/brokered sites). Furthermore, in sites where goods and services that affected the nature 
of the program treatment were donated to the sponsoring agency, the analysis estimated the market 
value of those contributions and counted it, too, as a program cost.? For these reasons, the costs 
presented here may differ from those reflected in a sponsoring agency's own fiscal records. 

This appendix does not present estimates of the cost of education and training services 
received by members of the control group. Thus, it provides no insights into the incremental 
investment that the JOBSTART sites made for the experimental group. An estimate of these 
incremental or “net” costs is incorporated in the benefit-cost analysis presented in Chapter 7. In that 
analysis, the value of experimental-control differences in earnings and other outcomes (the 
“benefits") is compared with the experimental-control differences in the cost of services producing 


This appendix was written by Barbara L. Fink. 

1The cost estimates in this appendix were generated as part of the 24-month impact analysis (Cave and 
Doolittle, 1991) and consequently reflect participation during only the first two years of follow-up. However, 
since the experimental group received virtually no JOBSTART services after the first two years, no additional 
cost data were collected for this report. This decision is reflected in both this appendix and the benefit-cost 
analysis in Chapter 7. 

example, at Chicago Commons, many of the basic supplies essential to operating some of the 

training courses were donated to the program. The estimated value of these supplies, as reported in the 
agency’s annual audit report, was thus counted as a program cost. As another illustration, the life skills 
workshops at both Chicago Commons and Conneliey in Pittsburgh were comucted free of charge at the 
program site by outside organizations. These donated services were thus valued and included in the total 
cost. Their estimated value was based on the number of sessions conducted and by a proxy value of the 
average cost per session to the agency providing the service. 
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those benefits. As Chapter 7 shows, there is virtually no difference between the net costs of 
JOBSTART and the gross costs reported here. 


A. Data Sources and Accounting Periods 
Data for the cost analysis were gathered from a variety of sources. These include: 
¢ individual staff salary information; 


¢ site expenditure reports, which showed overall expenditures on salaries and 
fringe benefits, rent, utilities, supplies, equipment, administration, and so on; 


* program enrollment and participation data covering JOBSTART and non- 
JOBSTART participants, both for the program as a whole and for individual 
components (such as education classes and training classes); 


¢ JTPA expenditure data in sites where JTPA funds were used to provide 
program services; 

* agency data on support service expenditures covering needs-based payments, 
transportation, food, child care, and other participant payments; 


¢ interviews with program staff concerning the allocation of staff time across 
program components and between JOBSTART and non-JOBSTART functions, 
and other aspects of site operations that affected the use of resources; and 


¢ MDRC’s MIS data on the experimental group's degree of participation in 
JOBSTART activities during the first 24 months of follow-up, as discussed 
above. 

In most cases, data from these sources covered a one-year “steady-state” period sometime 
between 1985 and 1988 (depending on the site), the years during which JOBSTART was funded.° 
However, the actual calendar months of this accounting period varied according to each site’s date 
of entry into the demonstration and the particular months covered by its annual fiscal reporting 
period.‘ 

Ideally, a steady-state period should reflect a time during which program operations are 
relatively stable. Although it was difficult to define such a period for JOBSTART because of the 
demonstration’s relatively short duration, the period selected in most sites began at least several 
months after the initiation of the project (in order to avoid the start-up costs associated with 


3The JOBSTART program at Connelley in Pittsburgh and SER/Corpus Christi changed substantially from 
the first year of operations to the next. Consequently, cost and participation data for both years were used. 
‘in some sites, participation data and expenditure reports did not cover exactly the same time period, so 

a number of additional adjustments had to be made in estimating average steaty-state expenditures. 
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beginning a new program) and ended at least several months prior to the termination of the 
demonstration (in order to exclude the phasedown period).” To remove the influence of inflation 
resulting from the use of costs from different calendar periods in different sites, all estimates were 
inflated or deflated to 1986 dollars. 


B. Excluded Costs 


In estimating the average cost per JOBSTART experimental, adjustments were made to 
exclude two categories of expenditures embedded in the sites’ fiscal data: (1) research-related costs 
and (2) the costs of services or activities that were offered to or used by non-JOBSTART 
participants. A fraction of program expenditures during the steady-state period resulted exclusively 
from research requirements. These included the extra costs involved in recruiting and processing 
individuals who became part of the control group, as well as the costs of staff time spent on 
conducting random assignment, completing the research enrollment forms, and participating in 
research-related interviews with MDRC personnel. These activities were not part of the normal 
effort of operating a JOBSTART program. Thus, the resources spent on them were not counted 
in the average cost estimates reported here. 

Several of the sites also offered a number of services that were not a part of the JOBSTART 
program but were nonetheless captured in the agencies’ aggregate expenditure reports. These too 
were excluded from the estimates of JOBSTART costs. This issue was most significant in the three 
Job Corps sites, where some Corpsmembers lived in dormitories at the centers, while others lived 
at home while attending Job Corps activities. All of the JOBSTART experimentals were 
nonresidents in these sites and were thus unaffected by the services intended exclusively for the 
residents. These residential-only services included: dormitory provisions, most nighttime and 
weekend recreational activities, and supervision by residential advisors and dormitory attendants. 
The share of total Job Corps costs associated with exclusively residential aspects of the program was 
thus estimated and eliminated from the resources counted in determining the cost of JOBSTART.’ 


SBecause the core education and training services in most sites were already in place prior to JOBSTART 
and continued after the demonstration, start-up and phasedown costs for the JOBSTART Demonstration 
‘were not an issue for these components. 

Saithough nonresidents were invited to participate in all recreation activities, they did so much less 
frequently than residential Corpsmembers. 

A technical assistance project at BSA in New York City is another example of separate activities whose 
costs were excluded in estimating JOBSTART costs. 


C. Calculating the Average Cost per Experimental 
In each site, the total average cost of JOBSTART per experimental was determined by 


summing the average cost of several relatively distinct program components and services. 
Determining these component costs involved several steps. First, an average unit cost during the 
steady-state period — that is, the cost of serving one person in the component for a specified unit 
of time — was calculated. The unit of measure varied for some components, mostly depending on 
whether the activity operated on an open-entry/open-exit or a fixed-cycle basis.® Thus, for open- 
entry components, the average cost of serving one person for one month in the activity was 
estimated; for fixed-cycle activities, the average cost per person who ever entered a given cycle of 
the activity was estimated. 

The numerator in these unit costs incorporated total expenditures for personnel and overhead 
functions, including expenses incurred for non-JOBSTART participants in sites where the 
experimentals were enrolled along with other persons in regular agency activities.” The value of 
donated goods and services was also counted as a program expenditure. The denominator included 
all participants (both JOBSTART and non-JOBSTART) in the component. Thus, for example, the 
unit cost of basic education at a site is the full cost of classes in which JOBSTART participants were 
enrolled, divided by the total number of students in the classes. 

The average unit cost was then multiplied by a corresponding participation measure.’ For 
Open-entry components, the unit cost was multiplied by the average number of months in which 
experimentals spent any hours in the activity (including zero months for experimentals with no hours 
in the activity)."’ For components that operated on a fixed-cycle basis, the unit cost (the cost per 
person who entered the activity) was multiplied by the proportion of JOBSTART experimentals who 


®When participants leave open-entry activities, they are typically replaced by other individuals. Thus, the 
average value of resources expended per person for these components varies with the leng:h of time an 
average participant receives that service. However, if entry into a training class is based on a fixed cycle and 
a student who drops out is not replaced by another student, the costs for that student's “slot” are still incurred 
by the agency on that student’s behalf, regardless of his or her length of stay. 

%For some activities, sites mainstreamed JOBSTART participants with non-JOBSTART participants. 
Other activities (for example, counseling, life skills instruction, basic education, or training services that were 
not normally provided as part of the agency's program) included JOBSTART participants only. See Auspos 
et al., 1989, for more details on the adaptations the sites made for JOBSTART. 

1In order to spread average unit costs among all experimentals and to cover the full period of their 
involvement in the program, the participation measures captured participation that occurred at any time 
during the demonstration, not jut within the steady-state period. 

This approach (that is, multiplying the average cost of serving one person for one month by the average 
number of months that JOBSTART experimentals spent in the component) allocates costs between 
JOBSTART and non-JOBSTART participants on the basis of their respective lengths of stay in the activity. 


ever entered the activity.'2, The values for both types of participation measures were based on 


the experiences of the experimentals in the 24-month impact sample, not of all experimentals who 
were randomly assigned.! 


Il. Accounting for Site Variations in Average Costs 


The average total cost of JOBSTART's core components per experimental varied widely across 
the sites. (See Table C.1.) Although it fell within $4,500 to $6,500 in most sites, it ranged from less 
than $2,100 in CET/San Jose, EGOS in Denver, and SER/Corpus Christi to a high of about $7,500 
in BSA in New York City. Several factors account for this diversity. Most notably, the sites differed 
in terms of both the amount of experimentals’ participation in JOBSTART and the unit cost of that 
participation. This can be seen in Table C.1, which presents information for each site on the 
average number of months that the experimental group participated in program activities and the 
average monthly cost of participation.'* Sites where the value of both of these variables was lower 
than in other sites were among the least expensive JOBSTART programs. For example, this 
combination of factors helps to explain why CET/San Jose (where experimentals participated for 
only 4.4 months and the average monthly cost was only $462) had the least expensive JOBSTART 
program. 

In several sites, higher unit costs were somewhat offset by shorter participation, yielding lower 
total average costs than were observed in some other sites. For example, the average monthly cost 
of JOBSTART at the Atlanta Job Corps was higher than at Connelley in Pittsburgh ($845 compared 
to $566). However, Atlanta’s overall average cost was lower ($4,173 compared to $5,185) because 
its experimentals were active in the program for less time (4.94 months compared to 9.16 months). 

The wide variation in average monthly costs across the sites (ranging from $303 in EGOS in 
Denver to $1,569 in BSA in New York City) has a number of sources. One is enrollment levels. 


12This approach assumes that there was no difference in the average cost of serving JOBSTART and non- 
JOBSTART participants who actually began the component. The costs were considered fxed, whether or 
not the students stayed until completion. No data were available for comparing the lengths of stay of 
JOBSTART and non-JOBSTART participants. 

Cave and Doolittle, 1991, for a discussion of the 24-month impact sample. In general, the 
eiciee in this sample, on average, had slightly more months with any hours of participation than did 
all tals, and thus may have been slightly more expensive to serve than those not in this sample. 

average monthly cost for each site — created to facilitate comparisons ac-oss sites — was calculated 
by dividing the average total cost (for experimentals) by the average number of months active in JOBSTART 
(for experimentals). Although this calculation assumes that the average total cost in all sites was variable, 
as noted earlier, the costs of fixed-cycle activities were actually calculated on a fixed-cycle basis. 
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TABLE C.1 


AVERAGE MONTHLY AND TOTAL COSTS PER EXPERIMENTAL AND PER PARTICIPANT, BY SITE 


Experimentais 
with Any Hours 
of Participation 

She (%) 
Concurrent 
Atlanta Job Corps 84.80 
CET/San Jose 64.00 
Chicago Commons 91.90 
Connetley h) 96.90 
East LA Skills Center 82.40 
EGOS (Derver) 93.55 
Phoenix Job Corps 86.57 
SER/Corpus Christi 96.30 
Sequential/in-house 
El Centro (Dallas) 100.00 
LA Job Corps 79.10 
Sequential/brokered 
Allentown (Buffalo) 100.00 
BSA (NYC) 75.38 
CREC (Harttord) 86.89 


Average Number of Months 


8.88 
477 
5.60 


8.68 
6.33 
6.30 


Average Cost per 
Month Active in 


SOURCE: MODRC calculations from site and MDRC participation, fiscal, and administrative data. 


Average Total Cost 
Cor 


5,166 


Feo eo gag re 


4,921 
3,178 
7,048 
§,243 
5,931 
2,219 
5,725 
2,134 


5,306 
7,016 


5,862 
9,928 
§,812 


NOTES: Estimates in this table used data for all experimentais for whom there were 24 months of follow-up survey data, including those who were 


assigned to JOBSTART but did not participate. "Participants" are the subset of experimentals who were active for at least one hour in any JOBSTART component 


within 24 months of random assignment. 


All costs are in 1986 dollars. 


®Uniess otherwise stated, the number of months active in JOBSTART Is defined as the number of months with hours in any JOBSTART component. 
estimates were obtained by dividing the average total cost per experimental by the percentage of experimentals with any hours of participation. 


°F or consistency with the definition of unit costs in this site, the number of months active in JOBSTART Is measured from the month of random assignment 
to the last month with hours in any component. 
SEo¢ consistency with the definition of unit costs in this site, the number of months active in JOBSTART is measured from the first month with hours in any 


JOBSTART component to the last month with hours in eny component. 
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For example, if the number of participants "on board" a program in a typical month is high relative 
to the number of program instructors, the total monthly instructional costs (and the corresponding 
overhead expenditures) will be spread over many people, lowering the average unit cost per 
participant. This factor helps to account for the relatively low monthly cost of JOBSTART at 
EGOS in Denver, a large public vocational school with more than 15,000 students. In contrast, at 
BSA in New York City, high monthly costs were partly the result of its having enrolled only about 
half the number of students the school had the capacity to serve at any one time. Staffing decisions 
can also affect costs. For example, Chicago Commons assigned two instructors to all training classes, 
an unusual practice among the JOBSTART sites, and this raised its average monthly cost per 
participant. Differences in wage scales further explain some of the variation in monthly costs. As 
an illustration, the average hourly wage paid to instructors at SER/Corpus Christi was about half the 
hourly rate received by teachers at the East Los Angeles Skills Center. 

Differences in the scope of activities and services across the sites also account for differences 
in average monthly costs. For instance, as will be seen below, the three least expensive sites had 
no life skills or work-readiness instruction, and one of them (CET/San Jose) spent little on support 
service payments and basic education as a separate activity. Differences in overhead costs, such as 
those for rent and administration, also varied across the JOBSTART sites. 

Table C.1 shows that in some sites a substantial proportion of experimentals left the program 
after random assignment and so never entered any program component. At the East Los Angeles 
Skills Center, for example, only 82 percent of the experimental group ever received JOBSTART 
services. One consequence of such attrition is that a site’s average cost per person actually served 
by the program is higher than its average cost per experimenal.5 At the Los Angeles Job Corps, 
it was 26 percent higher ($7,016 compared to $5,550). Although the average cost per experimental 
would be the appropriate number to include in a benefit-cost analysis for the JOBSTART 
evaluation, the average cost per participant may be a better guide for administrators interested in 
the implications for an agency's budget of operating a JOBSTART program. 


Til. Component Costs 


This section discusses how the costs of the individual JOBSTART components contributed to 
the total average cost at each site and further illustrates the sources of variation in the total costs 
across the sites. 


'SThe average cost per participant was calculated by dividing the average cost per experimental by the 
percentage of experimentals with any hours of participation. 
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A. Definitions of Components 


For purposes of the cost analysis, JOBSTART functions were divided into eight main 
components. The category of recruitment, intake, and orientation was defined to encompass sites’ 
efforts to attract and enroll individuals into the JOBSTART program and to prepare them, through 
special presentations or workshops, for attending the agency's regular education and training classes. 
This process involved screening applicants to determine whether they met all JOBSTART as well 
as JTPA or Job Corps eligibility criteria! Random assignment, special data collection, and the 
additional efforts devoted to recruiting and processing extra individuals to allow for the creation of 
a control group also occurred during the recruitment and intake stages. As previously mentioned, 
these latter activities were defined as research-only costs, and hence they were not counted in the 
average cost of this component. 

Following orientation, experimentals in all sites were scheduled to attend basic education 
classes. Occupational skills training classes were offered concurrently or following the completion 
of basic education. The costs of these two components were estimated separately, although in some 
sites the line between them was not sharp.!7 

Several sites also enriched their programs by offering work-readiness classes or life skills 
workshops that covered topics such as work habits, health, and financial management. In addition, 
the Job Corps sites offered avocational classes in driver education, sewing, and physical education. 
These were counted as part of the work-readiness/life skills component for the cost analysis. 

Job placement was defined to include instruction in job-seeking techniques as well as direct 
placement efforts. Coordination and counseling include staff efforts to monitor participants’ 
attendance and progress in JOBSTART activities and to counsel them on an as-needed basis. In 
a number of sites where JOBSTART was operated alongside other programs, a special counselor 
was designated to perform this function exclusively for the experimental group. 


16The costs included here for JTPA eligibility determination only cover a site's efforts to help applicants 
identify and collect the necessary documents and complete the required paperwork as part of the application 
process. They generally do not include the time that JTPA staff spent reviewing those documents and 
approving the applications. 

1 Data limitations have precluded perfect consistency across all sites in the definition of each component. 
Especially problematic is the distinction between basic education and skills training at CET/San Jose and 
Chicago Commons. To a large extent, basic education instruction in these two sites was integrated with 
occupational skills training. However, these sites also operated separate remedial education classes. The cost 
analysis counted only participation in those remediation classes as basic education. This definition of basic 
education is consistent with that used in the calculation of education hours, as reported in Chapter 3. 
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Support services were defined as the special expenditures intended to help motivate participants 
to attend program activities regularly, or to help offset some of the potential barriers to attendance. 
The particular types of support services that were available varied across sites, but included 
payments for child cave, transportation, needs-based payments, food, and attendance and 
achievement awards. 

Medical and dental services were an additional component offered in the three Job Corps sites 
through an on-site clinic. To a much lesser extent, such services were also offered at EGOS in 
Denver through a formal agreement with an outside agency to which staff routinely referred 
participants.!® These services are not considered to be part of the core JOBSTART model, 
however. Consequently, their costs have not been included in the total average costs reported above 
(although they have been estimated). 

It should be noted that the information used in allocating total site costs across components 
(such as the proportion of staff time spent on recruitment and intake versus counseling and 
coordination) was often imprecise. As a result, some component cost estimates are much less 
certain than others. However, this does not affect the estimate of any site’s average total cost. 


B. V: (+) e} 


Table C.2 shows the estimated average cost per experimental of each JOBSTART component 
for each site. In addition, for the four sites where medical and dental services were provided, the 
table shows how the total average cost changed when expenditures for these services were counted. 
In the three Job Corps sites, these amounted to fairly sizable expenditures — $564 per experimental 
at Los Angeles, $690 at Atlanta, and $357 at Phoenix. In addition, EGOS in Denver, which, 
through routine referrals, provided eyeglasses and dental examinations, spent $24 per experimental 
on medical expenses. 

Recruitment, intake, and orientation activities accounted for between 9 and 13 percent of the 
total average cost of the core JOBSTART components in most of the sites, but reached as high as 
19 percent at the East Los Angeles Skills Center. In absolute value, BSA in New York City spent 
the most on these upfront efforts ($1,313), while SER/Corpus Christi spent the least ($227 per 
experimental). 


18Some of the other JOBSTART sites also referred participants to such services at outside agencies, but 
on a much less formal basis. In these sites, the costs incurred by those agencies were not estimated. 
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TABLE C.2 
AVERAGE JOBSTART OPERATING COSTS PER EXPERIMENTAL, BY COMPONENT AND SITE 


East LA SER/ 
Atlanta CEeT/ Chicago Connetiey Skills EGOS Phoenix Corpus 
Job Corps San Jose Commons (Pittsburgh) Center (Denver) Job Corps Christi 
Component (3) ($) (8) (3) (3) ($) ($) ($) 
Recruttment, intake, 
and orientation 245 327 445 923 245 645 227 
Basic education 529 88 1,400 644 1,114 384 939 632 
Occupational skilts 
training 263 1,031 2,931 793 1,531 297 1,446 533 
Work-readiness or 
fe skilis training 639 wa 35 283 ‘a a 343 na 
Job developmem and 
placeniem assistance 92 308 262 334 % 19 168 73 
Counseling and program 
coordination 1,031 301 1,159 2,239 1,196 664 498 633 
Support services* 797 61 363 447 87 467 897 593° 
Subtotal for core 
JOBSTART components 4,173 2,034 6,477 5,185 4,887 2,076 4,956 2,098 
MedicaVdental® 690 ‘va Wa va wa 24 357 Wa 
Total 4,063 2,034 6,477 5,185 4,887 2,100 §,313 2,098 
(continued) 


354) 


Total §,306 6,114 5,862 7,484 5,166 


SOURCE: MORC calculations from site and MORC participation, fiscal, and administrative data. 
NOTES: Estimates in this table used data for all experimentals for whom there were 24 months 


inciudes such costs a8 needs-based and incentive payments; transportation, child care, and 


clothing ; and food. 
of data limitations, the support services cost for JOBSTART and non-JOBSTART 


Basic education, skills training, and coordination and counseling were usually among the most 


expensive components to operate across the sites. When taken together, they accounted for no less 
than 44 percent (the estimate for the Atlanta Job Corps) of the total average cost of the core 
JOBSTART components, and they reached as high as 86 percent (the estimate for SER/Corpus 
Christi). In most of the sites, this combination of functions accounted for at least two-thirds of the 
total average cost. 

Particularly notable are the high costs of basic education at the three sequential/brokered sites. 
At BSA in New York City and CREC in Hartford, basic education alone accounted for half the 
total cost ($3,836 and $2,634 per experimental, respectively). The cost of basic education was also 
high at Chicago Commons ($1,400), which added a separate education class specifically for 
JOBSTART participants, and at El Centro in Dallas ($1,301) and the East Los Angeles Skills 
Center ($1,114). Overall, the percentage of total average costs devoted strictly to basic education 
ranged from 4 percent at CET/San Jose to 51 percent at BSA and CREC. Education costs were 
especially low at CET/San Jose ($88 per experimental), in part because most of the hours that 
experimentals spent in that site were in training classes, which, it should be recalled, also included 
some work on basic education skills. (Basic education that occurred in the context of occupational 
skills training was counted as skills training.) 

The resources spent on skills training also varied widely across the sites. Not surprisingly, as 
a proportion of total average costs, expenditures for this component were lowest at the 
sequential/brokered sites (accounting for less than 10 percent of those costs), where only about one- 
quarter of the experimentals made the transition to a training activity. In the other sites, this 
component accounted for between 7 and 51 percent of total costs. In absolute value, it was most 
expensive at Chicago Commons ($2,931), where the training classes were small and operated on a 
fixed-cycle basis (dropouts within a cycle were usually not replaced by other students). The classes 
also involved fairly technical instruction and used significant amounts of purchased and donated 
supplies. 

Across all sites, the per-experimental cost of coordination and counseling ranged from $301 
at CET/San Jose to $2,239 at Connelley in Pittsburgh, where it accounted for an unusually high 43 
percent of the total average cost per experimental. In contrast, job search assistance and placement 
in most sites accounted for no more than 6 percent of total average costs, although it ranged from 
$19 per experimental at EGOS in Denver to $639 at El Centro in Dallas. In most sites, this tended 
to be a relatively inexpensive component because relatively few staff members were usually assigned 
to instruct or assist many different participants. (In two of the three sequential/orokered sites, job 
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search assistance was not offered by the sponsoring agency but, instead, was a function expected to 
be performed by the outside training vendors.) 

The sites also varied widely in their expenditures on work-readiness and life skills instruction. 
Indeed, some sites did not offer these activities at all, while others gave them considerable emphasis. 
Allentown in Buffalo, for example, spent more per experimental on life skills than it did on 
education ($1,438, or 25 percent of total average costs). 

Expenditures on support service costs ranged from $61 per experimental at CET/San Jose to 
$1,389 (or 25 percent of the total average cost) at the Los Angeles Job Corps. The other Job Corps 
sites, as well as SER/Corpus Christi, also devoted a relatively high proportion of resources to these 
services — about one-fifth to more than one-fourth of total average costs. 

Table C.3 shows the breakdown of support services costs for child care, transportation, food, 
needs-based payments, and other purposes (which included incentives for attendance and 
achievement). Almost all the sites offered needs-based payments,!? while about half of them paid 
for child care and transportation. Food costs were substantial in the three Job Corps sites (ranging 
from $153 to $259 per experimental), where the on-site cafeterias offered regular meals to both 
residential and nonresidential participants. 


19j1 Job Corps participants (including those in JOBSTART) received a $40 monthly living allowance, 
which increased to $60 after they were active for 61 ("good") days, and increased again to $80 after they were 
active for 181 days. After that, participants were eligible to receive a merit pay level of $90 to $100 per 
month. In addition, upon termination from the Job Corps, those who remained in the program for a 
specified length of time received a separate “readjustment allowance® for each month of participation. This 
allowance ranged from $75 to $100 per month depending on the number of days they were active. 
Participants could have a portion of this allowance sent to a dependent family member. If they made that 
choice, the Job Corps contributed an equal amount to the family member. 
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TABLE C.3 
AVERAGE JOBSTART SUPPORT SERVICES AND PARTICIPANT PAYMENT 


COSTS PER EXPERIMENTAL, BY SITE 
Concurrent 
East LA SER/ 
Atlanta CET/ Chicago Conneliey Skilis EGOS Phoenix Corpus 

Job Corps San Jose Commons (Pittsburgh) Center (Denver) Job Corps Christi 
Support Service (S) (8) (3) (8) (8) (8) (8) (S) 
Child care Wa va Va 107 21 339 na 86 
Transportation Wa Wa Wa va 12 121 Wa Wa 
Food 153 va na na 17 7 221 na 
Needs-based and Incentive 
payments, clothing 
allowances, and 
miscellaneous 644 61 363 340 37 0 676 507 
Total 797 61 363 447 87 467 897 593 
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TABLE C.3 (continued) 


Total 512 1,389 734 205 232 


SOURCE: MDRC calculations from site and MORC participation, fiscal, and administrative data. 
NOTES: Estimates in this table used data for all experimentals for whom there were 24 months 


of follow-up survey data, including those who were assigned to JOBSTART but did not participate. 
All costs are in 1986 collars. 
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TABLE 0.1 


MONTHLY EARNINGS FOR THE FULL SAMPLE, 


BY RESEARCH STATUS 


Outcome and Follow-Up Period 


Experimentals Controls 


($) 


312 
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TABLE D.1 (continued) 


SOURCE: MDRC calculations from JOBSTART enrollment form and survey data. 


NOTES: Calculations for this table used data for all sample members for whom there were 48 months of 
follow-up survey data, including those with values of zero for outcomes and those who were assigned to 
JOBSTART but did not i 

Random assignment did not always take place on the first of the month. For some sample members, the 
month of random assignment may be less than a month, beginning with the date of random assignment and 
ending on the last day of the month. 

Average experimental and control group outcomes reported here are adjusted means trom linear analysis of 

corarolling for differences in characteristics before random assignment (see Ostie, 1975, p. 
461; Cave, 1967; and Appendix B). There may be slight discrepancies in reported sums and differences of these 
adjusted means because of ; 

®The column labeled ‘p* is the statistical significance level of the difference between experimental and contro! 
group outcomes: that is, p is the probability that average outcomes are different only because of random error. A 
two-tailed t-test was applied to each difference between average experimemal and contro! group outcomes. 
Statistical significance levels are indicated as *** = 1 percent; ** = § percent; * = 10 percent. 
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TABLE 0.2 


MONTHLY EARNINGS FOR WOMEN LIVING WITH THEIR OWN 
CHILD(REN), BY RESEARCH STATUS 


Experimentals Controls Difference 

Outcome and Follow-Up Period ($) (Ss) (S) p? 

Earnings 
Month 1 15 26 11 0.226 
Month 2 29 64 36° 0.019 
Month 3 43 75 32e¢ 0.045 
Momth 4 64 78 14 0.460 
Month 5 75 81 4 0.752 
Month 6 80 109 29 0.194 
Month 7 94 107 -13 0.863 
Month 8 112 113 “1 0.970 
Month 9 123 129 4 0.626 
Month 10 122 132 -10 0.706 
Month 11 126 130 4 0.688 
Month 12 133 117 16 0.525 
Month 13 135 100 35 0.186 
Month 14 178 123 §5° 0.075 
Month 15 183 142 41 0.196 
Month 16 178 145 KX] 0.276 
Month 17 160 189 21 0.512 
Month 18 181 163 18 0.881 
Month 19 181 179 2 0.961 
Month 20 191 179 12 0.725 
Month 21 178 190 12 0.781 
Month 22 175 196 -21 0.564 
Month 23 172 185 13 0.706 
Month 24 164 186 22 0.523 
Month 25 189 194 35 0.262 
Month 26 175 209 xu 0.327 
Month 27 201 195 6 0.878 
Month 28 215 193 2 0.527 
Month 29 220 179 ai 0.283 
Month 30 229 183 4 0.206 
Month 31 239 188 51 0.170 
Month 32 246 178 68* 0.072 
Month 33 236 187 49 0.205 
Month 34 246 203 43 0.282 
Month 35 263 227 % 0.378 
Month 36 272 236 % 0.386 
Month 37 261 234 ri 0.803 
Month 38 254 235 19 0.635 
Month 39 247 235 12 0.767 
Month 40 258 225 33 0.414 
Month 41 254 222 32 0.426 
Month 42 269 228 a 0.326 
Month 43 266 246 20 0.632 
Month 44 274 251 23 0.887 
Month 45 260 243 17 0.606 
Month 46 261 225 % 0.375 
Month 47 27 242 35 0.405 
Month 48 266 270 4 0.915 

Sample size 287 251 

SOURCE AND NOTES: See Tabie D.1. 
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TABLE D.3 


MONTHLY EARNINGS FOR WOMEN NOT LIVING WITH THEIR OWN 


CHILD(REN), BY RESEARCH STATUS 


Experimentais Controls Difference 

Outcome and Follow-Up Period ($) (8) (8) p® 
Earnings 
Month 1 35 80 -15 0.187 
Month 2 BB 124 -38° 0.071 
Month 3 4 135 «41* 0.087 
Month 4 102 181 age 0.032 
Month 5 122 177 -65°* 0.023 
Month 6 130 176 46° 0.068 
Month 7 144 191 47* 0.071 
Month 8 163 204 41 0.126 
Month 9 185 182 3 0.920 
Month 10 198 190 8 0.71 
Month 11 219 227 8 0.827 
Month 12 218 235 -17 0.634 
Month 13 220 211 9 0.780 
Month 14 266 225 4 0.218 
Month 15 270 234 36 0.281 
Month 16 265 261 4 0.925 
Month 17 259 284 25 0.467 
Month 18 260 297 37 0.286 
Month 19 257 300 43 0.239 
Month 20 262 287 25 0.488 
Month 21 313 238 15 0.685 
Month 22 317 282 35 0.378 
Month 23 321 291 30 0.445 
Month 24 335 299 36 0.380 
Month 25 336 2r1 65 0.101 
Month 26 350 310 40 0.341 
Month 27 356 330 26 0.551 
Month 28 350 330 11 0.796 
Month 29 355 347 8 0.854 
Month 30 347 338 9 0.835 
Month 31 350 351 1 0.987 
Month 32 353 346 7 0.876 
Month 33 368 330 29 0.491 
Month 34 373 315 88 0.167 
Month 35 384 901 @3° 0.058 
Month 36 386 904 e2° 0.066 
Month 37 389 304 es* 0.056 
Month 38 301 927 64 0.186 
Month 39 301 “4 47 0.314 
Month 40 389 349 ry) 0.380 
Month 41 377 335 42 0.349 
Month 42 377 KY.) 28 0.544 
Month 43 400 362 38 0.423 
Month 44 305 385 40 0.390 
Month 45 384 957 27 0.565 
Month 46 383 352 31 0.498 
Month 47 966 349 17 0.712 
Month 48 332 328 4 0.926 

Sample size 283 250 


SOURCE AND NOTES: See Table 0.1. This subgroup includes women who did not have children. 
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TABLE 0.4 


MONTHLY EARNINGS FOR MEN, BY RESEARCH STATUS 


Controls Difference 

Outcome and Follow-Up Period (S) ($) (S) p® 

Earnings 
Month 1 73 114 aiee 0.015 
Month 2 156 228 -72ee8 0.006 
Month 3 174 259 85°** 0.001 
Month 4 190 292 -102°¢* 0.000 
Month 5 210 319 -109¢ee 0.000 
Month 6 228 328 -100°¢* 0.001 
Month 7 271 $24 53° 0.066 
Month 8 298 320 2 0.461 
Month 9 318 365 47 0.154 
Month 10 330 377 47 0.147 
Month 11 320 403 83°"" 0.009 
Month 12 361 412 51 0.117 
Month 13 358 391 33 0.286 
Month 14 415 44 -19 0.586 
Month 15 433 457 24 0.481 
Month 16 440 472 32 0.374 
Month 17 440 488 48 0.178 
Month 18 446 475 29 0.402 
Month 19 450 495 45 0.204 
Month 20 474 503 -29 0.419 
Month 21 492 506 -14 0.683 
Month 22 490 $32 42 0.244 
Month 23 495 $32 37 0.311 
Month 24 504 546 42 0.258 
Month 25 553 824 29 0.468 
Month 26 625 553 Tee 0.091 
Month 27 638 548 90*¢ 0.033 
Month 28 637 856 ei* 0.062 
Month 29 609 854 55 0.199 
Month 30 626 877 49 0.260 
Month 31 627 584 “3 0.319 
Month $2 618 596 2 0.609 
Month 33 599 615 -16 0.708 
Month 34 600 619 -19 0.660 
Month 35 625 622 3 0.951 
Month 36 645 609 36 0.414 
Month 37 643 586 57 0.184 
Month 38 625 585 40 0.348 
Month 39 641 593 48 0.264 
Month 40 646 S04 52 0.221 
Month 41 687 606 51 0.225 
Month 42 666 620 46 0.274 
Month 43 657 600 87 0.183 
Month 44 635 589 6 0.306 
Month 45 616 897 19 0.651 
Month 46 620 580 40 0.338 
Month 47 600 872 28 0.508 
Month 48 893 $75 18 0.664 

Sample size 9688 953 


SOURCE AND NOTES: See Tabie 0.1. 
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TABLE 0.5 


MONTHLY EMPLOYMENT RATES FOR THE FULL SAMPLE, 
BY RESEARCH STATUS 


Experimentas § Comrois 


Outcome and Follow-Up Period (%) (%) Difference p® 
Ever employed 
Month 1 13.7 20.4 6.70" 0.000 
Month 2 16.6 24.9 B.4ee* 0.000 
Month 3 19.0 27.8 8.8°** 0.000 
Month 4 21.6 30.7 -9.2°¢* 0.000 
Month 5 24.0 33.1 9.190¢ 0.000 
Month 6 26.6 33.7 7.1908 0.000 
Month 7 30.8 34.5 3.8° 0.064 
Month 8 32.9 35.5 26 0.2068 
Month 9 348 96.7 1.9 0.367 
Month 10 37.4 38.3 0.9 0.657 
Month 11 . $7.3 40.6 3.3 0.121 
Month 12 40.4 42.0 -1.6 0.435 
Month 13 47.0 44.0 3.0 0.162 
Month 14 40.2 36.9 33 0.111 
Month 15 41.9 39.3 26 0.216 
Month 16 42.0 40.0 1.9 0.357 
Month 17 41.4 42.4 1.0 0.634 
Month 18 41.9 43.0 “1.1 0.600 
Month 19 415 43.7 22 0.296 
Month 20 42.3 43.4 “11 0.898 
Month 21 442 43.6 06 0.778 
Month 22 45.1 44.1 1.1 0.616 
Month 23 44.6 “49 03 0.902 
Month 24 45.3 “49 04 0.837 
Month 25 42.7 42.4 0.4 0.858 
Month 26 45.3 42.8 25 0.235 
Month 27 46.0 43.5 26 0.219 
Month 28 46.5 43.1 3.3 0.118 
Month 29 46.0 42.9 31 0.142 
Month 30 46.8 “45 23 0.287 
Month 31 46.9 45.7 1.2 0.560 
Month 32 47.1 45.4 1.8 0.407 
Month 33 46.8 45.4 1.4 0.815 
Month 34 46.5 46.2 0.3 0.873 
Month 35 48.4 46.1 23 0.278 
Month 36 48.1 45.0 3.1 0.146 
Month 37 49.2 45.6 35° 0.097 
Month 38 47.7 46.1 1.6 0.465 
Month 39 48.2 46.6 1.6 0.449 
Month 40 48.4 469 1.6 0.466 
Month 41 48.0 46.9 14 0.896 
Month 42 48.7 46.0 27 0.200 
Month 43 48.9 472 18 0.403 
Month 44 48.8 47.0 18 0.413 
Month 45 474 47.4 0.0 0.984 
Month 46 478 46.0 1.8 0.397 
Month 47 475 45.6 1.9 0.373 
Month 48 45.4 46.2 09 0.682 

Sample size 988 953 


Outcome and Follow-Up Period 


Sampie size 
SOURCE AND NOTES: See Table D.1. 


TABLE D.6 


MONTHLY HOURS WORKED FOR THE FULL SAMPLE, 
BY RESEARCH STATUS 


Experimentals Controls 


15.68 
34.16 
38.66 
42.89 
47.64 
48.69 
48.47 


Difference 


5.720* 
12,019* 
-12.95¢** 
-13,83°** 
-15,23°e* 
-13,76°** 
831° 
5.55° 
5.30 


TABLE 0.7 


MONTHLY WEEKS WORKED FOR THE FULL SAMPLE, 
BY RESEARCH STATUS 


Outcome and Follow-Up Period Experimentals Controls 


Number of weeks employed 
0.30 0.44 
0.65 0.96 
0.74 
0.85 
0.94 
1.01 
1.15 
1.27 
1.35 
1.41 
1.44 
1.55 
1.45 
1.62 


SOURCE AND NOTES: See Table 0.1. 


MONTHLY EMPLOYMENT RATES FOR WOMEN LIVING WITH THEIR 
OWN CHILD(REN), BY RESEARCH STATUS 


TABLE 0.8 


Controls 

Outcome and Follow-Up Period (%) (%) Difference p* 
Ever employed 
Month 1 62 87 25 0.290 
Month 2 64 122 -5.8°* 0.025 
Month 3 9.9 14.1 42 0.183 
Month 4 12.4 15.5 3.1 0.321 
Month 5 14.7 16.4 A? 0.611 
Month 6 16.6 18.1 “1.5 0.670 
Month 7 19.1 19.1 0.41 0.988 
Month & 21.6 21.4 02 0.957 
Month 9 23.6 23.3 03 0.929 
Month 10 25.5 23.3 23 0.854 
Month 11 23.6 23.3 03 0.943 
Month 12 24.3 23.8 0.5 0.894 
Month 13 30.2 23.2 7.0° 0.080 
Month 14 26.5 19.5 7.1° 0.063 
Month 15 26.8 228 40 0.300 
Month 16 27.6 21.6 6.0 0.113 
Month 17 26.5 23.9 26 0.802 
Month 18 26.5 24.7 1.9 0.637 
Month 19 26.3 26.1 0.1 0.971 
Month 20 24.7 26.9 21 0.868 
Month 21 23.4 27.4 40 0.303 
Month 22 23.6 26.9 33 0.996 
Month 23 24.0 25.2 12 0.760 
Month 24 23.2 23.7 05 0.807 
Month 25 20.0 27.0 -7.0° 0.060 
Month 26 21.8 27.5 $7 0.132 
Month 27 24.2. 26.2 2.1 0.800 
Month 28 26.0 25.6 04 0.918 
Month 29 27.2 24.0 32 0.411 
Month 30 27.2 24.7 25 0.820 
Month 31 29.5 25.2 43 0.281 
Month 32 28.8 24.7 40 0.311 
Month 33 278 23.7 3.8 0.335 
Month 34 272 26.3 08 0.832 
Month 35 29.3 29.0 03 0.943 
Month 36 32.0 27.4 45 0.263 
Month 37 92.0 28.6 33 0.417 
Month 38 29.8 30.3 08 0.850 
Month 39 29.7 29.8 0.1 0.963 
Month 40 31.5 27.9 37 0.364 
Month 41 90.1 278 23 0.568 
Month 42 31.1 27.5 36 0.370 
Month 43 30.2 29.7 05 0.600 
Month 44 31.6 29.4 21 0.607 
Month 45 29.9 29.2 0.7 0.887 
Month 46 $12 25.8 5.4 0.178 
Month 47 33.5 29.0 48 0.275 
Month 48 9322 90.4 18 0.656 

Sample size ona 953 

SOURCE AND NOTES: See Table 0.1. 
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TABLE 0.9 


MONTHLY HOURS WORKED FOR WOMEN LIVING WITH THEIR OWN 
CHILD(REN), BY RESEARCH STATUS 


Outcome and Follow-Up Period Experimentais Controls 


Total hours worked 
Month 1 


SOURCE AND NOTES: See Table 0.1. 
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TABLE 0.10 


MONTHLY WEEKS WORKED FOR WOMEN LIVING WITH THEIR OWN 
CHILD(REN), BY RESEARCH STATUS 


Outcome and Follow-Up Period Experimentals Controls Difference 

Number of weeks employed 
Month 1 0.13 0.17 0.05 
Month 2 0.27 0.46 0.19* 
Month 3 0.35 0.56 0.20* 
Month 4 0.47 0.58 0.11 
Month 5 0.57 0.61 0.04 
Month 6 0.60 0.72 0.13 
Month 7 0.70 0.73 0.04 
Month 8 0.64 0.80 0.04 
Month 9 0.68 0.92 0.04 
Month 10 0.89 0.91 0.02 
Month 11 0.69 0.91 0.02 
Month 12 0.90 0.68 0.02 
Month 13 0.63 0.66 0.17 
Month 14 1.04 0.77 0.27* 
Month 15 1.05 0.87 0.18 
Month 16 1.05 0.68 0.17 
Month 17 1.03 0.93 0.10 
Morth 18 1.04 0.99 0.06 
Month 19 1.03 1.04 0.01 
Month 20 1.01 1.05 0.04 
Month 21 0.93 1.07 0.14 
Month 22 0.93 1.07 0.14 
Month 23 0.92 0.98 0.06 
Month 24 0.87 0.97 0.10 
Month 25 0.82 1.13 0.32** 
Month 26 0.89 1.14 02 
Month 27 1.02 1.08 0.06 
Month 28 1.06 1.06 0.02 
Month 29 1.06 1.00 0.08 
Morth 30 1.18 1.03 0.13 
Month 31 1.20 1.06 0.15 
Month 32 1.19 0.99 0.20 
Month 33 1.14 0.98 0.16 
Month 34 1.13 1.06 0.08 
Month 35 1.22 1.13 0.08 
Month 36 1.27 1.18 0.00 
Month 37 1.26 1.17 0.00 
Month 38 1.18 1.20 0.01 
Month 30 1.21 1.21 0.00 
Month 40 1.26 1.18 0.11 
Month 41 1.23 1.14 0.09 
Month 42 1.26 1.13 0.13 
Month 43 1.24 1.18 0.06 
Month 44 1.29 1.19 0.10 
Month 45 1.25 1.13 0.12 
Month 46 1.26 1.03 0.23 
Month 47 1.34 1.18 0.19 
Month 48 1.3 1.24 0.06 

Sample size 257 251 


TABLE D.11 


MONTHLY EMPLOYMENT RATES FOR WOMEN NOT LIVING WITH THEIR OWN 
CHILD(REN), BY RESEARCH STATUS 


Experimentais Controls 
Outcome and Follow-Up Period (%) (%) 


SOURCE AND NOTES: See Table 0.1. This subgroup includes women who did not have children. 
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TABLE D.12 


MONTHLY HOURS WORKED FOR WOMEN NOT LIVING WITH 
THEIR OWN CHILD(REN), BY RESEARCH STATUS 


Outcome and Follow-Up Period Experimentals Controts Difference p* 

Total hours worked 
Month 1 8.71 11.69 2.98 0.253 
Month 2 20.33 29.07 814° 0.063 
Month 3 23.12 31.65 852° 0.069 
Month 4 24.40 34.70 -10.31%* 0.044 
Month 5 29.23 40.76 -11.83¢¢ 0.032 
Month 6 31.15 40.39 -9.23° 0.001 
Month 7 34.96 43.85 8.90 0.119 
Month 8 38.48 47.31 8.83 0.138 
Month 9 42.06 41.76 0.31 0.959 
Month 10 44.21 42.59 1.62 0.787 
Month 11 48.63 48.46 0.21 0.973 
Month 12 47.42 49.07 -1.64 0.764 
Month 13 46.22 42.92 3.30 0.545 
Month 14 $4.08 44.83 9.24 0.141 
Month 15 54.55 47.16 7.39 0.245 
Month 16 53.67 53.49 0.18 0.978 
Month 17 §2.80 57.56 4.76 0.464 
Month 18 53.60 59.33 6.73 0.385 
Month 19 53.12 $8.82 -§.70 0.306 
Month 20 $3.41 56.36 2.95 0.665 
Month 21 63.56 56.63 6.93 0.327 
Month 22 64.11 52.75 11.36 0.100 
Month 23 63.95 $4.84 9.10 0.195 
Month 24 64.96 56.58 8.38 0.235 
Month 25 59.17 49.00 10.16 0.129 
Month 26 62.41 $4.10 8.31 0.242 
Month 27 63.27 58.97 4.30 0.551 
Month 28 61.95 60.24 1.70 0.814 
Month 29 62.80 62.48 0.32 0.968 
Month 30 60.91 60.77 0.14 0.964 
Month 31 62.04 63.39 1.35 0.850 
Month 32 63.14 62.61 0.82 0.941 
Month 33 65.86 61.51 4.35 0.530 
Month 34 65.19 87.56 7.63 0.272 
Month 35 66.12 $6.02 10.10 0.184 
Month 36 67.11 $5.39 11.72 0.104 
Month 37 68.00 56.18 11.82 0.101 
Month 38 68.43 61.52 6.91 0.349 
Month 39 68.41 64.19 422 0.878 
Month 40 67.69 64.49 3.20 0.667 
Month 41 64.47 61.06 3.41 0.638 
Month 42 63.96 62.98 1.00 0.802 
Month 43 68.04 65.05 2.99 0.687 
Month 44 67.96 64.33 3.63 0.624 
Month 45 66.64 64.15 2.69 0.713 
Month 46 67.81 61.43 6.38 0.375 
Month 47 65.71 62.78 2.93 0.687 
Month 48 $9.49 $0.44 0.06 0.994 

Sample size 283 250 


SOURCE AND NOTES: See Table D.1. This subgroup includes women who did not have children. 


“38 410) 


TABLE 0.13 


MONTHLY WEEKS WORKED FOR WOMEN NOT LIVING WITH THEIR 
OWN CHILD(REN), BY RESEARCH STATUS 


Outcome and Follow-Up Period Experimentals Controls 


SOURCE AND NOTES: See Table D.1. This subgroup includes women who did not have children. 
-307- 


411 


TABLE 0.14 


MONTHLY EMPLOYMENT RATES FOR MEN, 
BY RESEARCH STATUS 


Experimentals Controls 
Outcome and Follow-Up Period (%) (%) Difference p® 


Outcome and Follow-Up Period 


Total hours worked 
Month 1 
Month 2 
Month 3 
Month 4 


TABLE D.15 


MONTHLY HOURS WORKED FOR MEN, 


BY RESEARCH STATUS 


Experimentais Controls 


23.47 
48.01 
54.37 
61.73 
68.10 
67.71 
65.07 


TABLE D.16 
MONTHLY WEEKS WORKED FOR MEN, 


BY RESEARCH STATUS 

Outcome and Follow-Up Period Experimentais Controls Difference p@ 

Number of weeks employed 
Month 1 0.39 0.64 0.24%¢* 0.00% 
Month 2 0.85 1.31 0.46*** 0.000 
Month 3 0.97 1.50 0.52*¢° 0.000 
Month 4 1.10 1.68 0.58°** 0.000 
Month 5 1.17 1.82 0.65*** 0.000 
Month 6 1.25 1.81 0.56°** 0.000 
Month 7 1.46 1.77 0.31% 0.019 
Month 8 1.56 1.78 0.22 0.112 
Month 9 1.69 1.93 0.23° 0.092 
Month 10 1.77 2.03 0.27° 0.052 
Month 11 1.74 2.13 0.39°** 0.004 
Month 12 2.00 2.17 0.17 0.202 
Month 13 1.84 1.94 0.10 0.453 
Month 14 2.01 2.03 0.02 0.893 
Month 15 2.12 2.11 0.01 0.941 
Month 16 2.13 2.15 0.02 0.881 
Month 17 212 223 0.11 0.422 
Month 18 214 2.25 0.10 0.448 
Month 19 2.15 2.32 0.18 0.209 
Month 20 222 234 0.12 0.360 
Month 21 231 2.35 0.04 0.763 
Month 22 231 2.47 0.16 0.250 
Month 23 2.32 2.48 0.16 0.248 
Month 24 2.41 251 0.10 0.488 
Month 25 239 2.38 0.01 0.951 
Month 26 253 2.42 0.11 0.425 
Month 27 257 2.30 0.18 0.200 
Month 28 2.54 2.40 0.14 0.311 
Month 29 24 2.42 0.02 0.891 
Month 30 2.82 251 0.01 0.967 
Month 31 254 2.52 0.02 0.892 
Month 32 253 2.83 0.00 0.983 
Month 33 249 2.60 0.11 0.419 
Month 34 250 2.60 0.11 0.446 
Month 35 251 2.60 0.09 0.516 
Month 36 2.56 2.56 0.01 0.959 
Month 37 284 2.80 0.04 0.771 
Month 38 247 2.50 0.03 0.811 
Month 39 2.54 2.82 0.02 0.873 
Month 40 2.56 2.53 0.03 0.622 
Month 41 261 256 0.05 0.736 
Month 42 2.67 2.85 0.12 0.396 
Month 43 259 2.51 0.08 0.845 
Month 44 253 2.54 0.00 0.972 
Month 45 248 2.85 0.07 0.616 
Month 46 2.50 2.82 0.02 0.892 
Month 47 241 2.45 0.05 0.726 
Month 48 2.37 2.46 0.09 0.819 

Sampie size 448 482 


SOURCE AND NOTES: See Table 0.1. 
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PROGRAMS FOR YOUTH 
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A test of a program combining education, training, support services, and job placement for very 
disadvantaged young high school dropouts. 
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Implementing JOBSTART: A Demonstration for School Dropouts in the JTPA System. 1989. Patricia Auspos, 
George Cave, Fred Doolittle, Gregory Hoerz. 


Assessing JOBSTART: Interim Impacts of a Program for School Dropouts. 1991. George Cave, Fred 
Doolittle. 


The School-to-Work Transition Project 
A study of innovative programs that help students make the transition from school to work. 


The School-to-Work Transition and Youth Apprenticeship: Lessons from the U.S. Experience. 1993. Thomas 
Bailey, Donna Merritt. 


The Career Beginnings Evaluation 
An evaluation of a program that seeks to increase college attendance and improve job quality among 
disadvantaged high school students. 


Career Beginnings Impact Evaluation: Findings from a Program for Disadvantaged High School Students. 
1990. George Cave, Janet Quint. 


The Youth Incentive Entitlement Pilot Projects (YIEPP) Demonstration 
A test of a school-conditioned job guarantee for low-income youth. 
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THE NATIONAL JTPA STUDY 
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PROGRAMS FOR TEENAGE PARENTS ON WELFARE 


The LEAP Evaluation 
An evaluation of Ohio's Learning, Earning, and Parenting (LEAP) Program, which uses financial 
incentives to encourage teenage parents on welfare to stay in or return to school. 


LEAP: Implementing a Welfare Initiative to Improve School Atiendance Among Teenage Parents. 1991. Dan 
Bloom, Hilary Kopp, David Long, Denise Polit. 

LEAP: Interim Findings on a Welfare Initiative to Improve School Attendance Among Teenage Parents. 1993. 
Dan Bloom, Veronica Fellerath, David Long, Robert Wood. 


The New Chance Demonstration 
A test of a comprehensive program of services that seeks to improve the economic status and 
general well-being of a group of highly disadvantaged young women and their children. 


New Chance: Implementing a Comprehensive Program for Disadvantaged Young Mothers and Their Children. 
1991. Janet Quint, Barbara Fink, Sharon Rowser. 


Redirection 
A test of a comprehensive program of services for pregnant and parenting teenagers. 


The Challenge of Serving Teenage Mothers: Lessons from Project Redirection. Monograph. 1988. Denise 
Polit, Janet Quint, James Riccio. 


The Community Service Projects 
A test of a New York State teenage pregnancy prevention and services initiative. 


The Community Service Projects: A New York State Adolescent Pregnancy Initiative. 1986. Cynthia Guy. 
The Community Service Projects: Final Report on a New York State Adolescent Pregnancy Prevention and 
Services Program. 1988. Cynthia Guy, Lawrence Bailis, David Palasits, Kay Sherwood. 


WELFARE-TO-WORK PROGRAMS 


From Welfare to Work (Russell Sage Foundation). Book. 1991. Judith M. Gueron, Edward Pauly. A 
synthesis of research findings on the effectiveness of welfare-to-work programs. Chapter 1, which is 
the summary of the book, is also published separately by MDRC. 

Reforming Welfare with Work (Ford Foundation). Monograph. 1987. Judith M. Gueron. A review of 
welfare-to-work initiatives in five states. 

Learning from the Voices of Mothers: Single Mothers’ Perceptions of the Trade-offs Between Welfare and 
Work. Working Paper. 1993. LaDonna Pavetti. 


Papers for Practitioners Series 


Assessing JOBS Participants: Issues and Trade-offs. 1992. Patricia Auspos, Kay Sherwood. Lessons from 
t and training programs for assessment in JOBS. 
Linking Welfare and Education: A Study of New Programs in Five States. 1992. Edward Parly, David Long, 
Karin Martinson. Key issues in providing education services to welfare recipients. 


An evaluation of welfare-to-work programs operating under the Job Opportunities and Basic Skills 
Training (JOBS) provisions of the Family Support Act of 1988. 
From Welfare to Work (Russell Sage Foundation). Book. 1991. Judith M. Gueron, Edward Pauly. See 
above. 


The Saturation Work Initiative Model in San Diego: A Five-Year Follow-up Study. 1993. Daniel Friedlander, 
Gayle Hamilton. 
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The GAIN Evaluation 
An evaluation of California's Greater Avenues for Independence (GAIN) Program, which is 
currently operating as the state’s JOBS program and features upfront basic education as well as job 
search and other activities. 


GAIN: Planning and Early Implementation. 1987. John Wallace, David Long. 

GAIN: Child Care in a Welfare Employment Initiative. 1989. Karin Martinson, James Riccio. 

GAIN: Early Implementation Experiences and Lessons. 1989. James Riccio, Barbara Goldman, Gayle 
Hamilton, Karin Martinson, Alan Orenstein. 

GAIN: Participation Patterns in Four Counties. 1991. Stephen Freedman, James Riccio. 

GAIN: Program Strategies, Participation Paners, and First-Year Impacts in Six Counties. 1992. James Riccio, 
Daniel Friedlander. 

GAIN: Two-Year Impacts in Six Counties. 1993. Daniel Friedlander, James Riccio, Stephen Freedman. 


The Saturation Work Initiative Model (SWIM) 
A test of the feasibility and effectiveness of an ongoing participation requirement in a welfare-to- 
work program. 
Final Report on the Saturation Work Initiative Model in San Diego. 1989. Gayle Hamilton, Daniel 
Friedlander. 


The Saturation Work Initiative Mode! in San Diego: A Five-Year Follow-up Study. 1993. Daniel Friedlander, 
Gayle Hamilton. 


Subgroup/Performance Indicator Study 
A study of the impacts of selected welfare-to-work programs on subgroups of the AFDC caseload. 
sg: oh eh ee a aaa ala ai ala a ac 1987. 
niel Friedlander, David Long. 
Subpoup Ingots end Peformance car for Slee Welfare Employment Programs. 1988. Daniel 


THE PARENTS’ FAIR SHARE DEMONSTRATION 
A demonstration aimed at reducing child poverty by increasing the job-holding, earnings, and child 
support payments of unemployed, noncustodial parents (usually fathers) of children receiving public 
assistance. 


Caring and Paying: What Fathers and Mothers Say About Child Support. 1992. Frank Furstenberg, Jr., Kay 
Sherwood, Mercer Sullivan. 
Child Support Enforcement Processes: A Case Study. Working Paper. 1993. Dan Bloom. 


THE NATIONAL SUPPORTED WORK DEMONSTRATION 
A test of a transitional work experience program for four disadvantaged groups. 


Summary and Findings of the National Supported Work Demonstration. Book. 1980. Board of Directors, 
MDRC. 

The Impact of Supported Work on Young School Dropouts. 1960. Rebecca Maynard. 

The Impact of Supported Work on Ex-Addicts. 1981. Katherine Dickinson, Rebecca Maynard. 

The Impact of Supported Work on Ex-Offenders. 1981. Irving Piliavin, Rosemary Gartner. 

The Impact of Supported Work on Long-Term Recipients of AFDC Benefits. 1981. Stanley Masters, Rebecca 


Maynard. 
The Supported Work Evaluation: Final Benefit-Cost Analysis. 1981. Peter Kemper, David Long, Craig 
Thornton. 
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